Oct 30 19:17:59.709165 ip-10-0-132-152 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Oct 30 19:17:59.709179 ip-10-0-132-152 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Oct 30 19:17:59.709188 ip-10-0-132-152 systemd[1]: kubelet.service: Failed with result 'resources'.
Oct 30 19:17:59.709527 ip-10-0-132-152 systemd[1]: Failed to start Kubernetes Kubelet.
Oct 30 19:18:09.852379 ip-10-0-132-152 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Oct 30 19:18:09.852420 ip-10-0-132-152 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot c43b134f0af34ca4adbe6b35b4b990c9 --
Oct 30 19:20:16.242148 ip-10-0-132-152 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 19:20:16.735304 ip-10-0-132-152 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 19:20:16.735304 ip-10-0-132-152 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 19:20:16.735304 ip-10-0-132-152 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 19:20:16.735304 ip-10-0-132-152 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 19:20:16.741855 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741734    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:20:16.741855 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741852    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:20:16.741855 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741874    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:20:16.741855 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741880    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:20:16.742072 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741884    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:20:16.742072 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741886    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:20:16.742072 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741889    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:20:16.742072 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741892    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:20:16.742072 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741895    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:20:16.742072 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741897    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:20:16.742072 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741901    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:20:16.742072 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741904    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:20:16.742072 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741907    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:20:16.742072 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741909    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:20:16.742072 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741912    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:20:16.742072 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741917    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:20:16.742072 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741922    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:20:16.742072 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741926    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:20:16.742072 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741928    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:20:16.742072 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741931    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:20:16.742072 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741935    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:20:16.742072 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741939    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:20:16.742072 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741942    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:20:16.742506 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741944    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:20:16.742506 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741948    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:20:16.742506 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741952    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:20:16.742506 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741955    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:20:16.742506 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741958    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:20:16.742506 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741961    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:20:16.742506 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741964    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:20:16.742506 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741966    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:20:16.742506 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741968    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:20:16.742506 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741971    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:20:16.742506 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741973    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:20:16.742506 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741977    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:20:16.742506 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741979    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:20:16.742506 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741982    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:20:16.742506 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741985    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:20:16.742506 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741993    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:20:16.742506 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741997    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:20:16.742506 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.741999    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:20:16.742506 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742002    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:20:16.742506 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742004    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:20:16.742977 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742007    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:20:16.742977 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742010    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:20:16.742977 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742012    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:20:16.742977 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742015    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:20:16.742977 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742018    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:20:16.742977 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742020    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:20:16.742977 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742023    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:20:16.742977 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742025    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:20:16.742977 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742028    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:20:16.742977 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742030    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:20:16.742977 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742033    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:20:16.742977 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742035    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:20:16.742977 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742037    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:20:16.742977 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742040    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:20:16.742977 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742043    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:20:16.742977 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742046    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:20:16.742977 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742048    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:20:16.742977 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742050    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:20:16.742977 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742053    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:20:16.743407 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742055    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:20:16.743407 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742058    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:20:16.743407 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742060    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:20:16.743407 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742062    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:20:16.743407 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742065    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:20:16.743407 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742068    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:20:16.743407 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742070    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:20:16.743407 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742074    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:20:16.743407 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742076    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:20:16.743407 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742079    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:20:16.743407 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742081    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:20:16.743407 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742084    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:20:16.743407 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742087    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:20:16.743407 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742089    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:20:16.743407 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742092    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:20:16.743407 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742094    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:20:16.743407 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742097    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:20:16.743407 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742099    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:20:16.743407 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742102    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:20:16.743852 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742104    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:20:16.743852 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742106    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:20:16.743852 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742110    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:20:16.743852 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742113    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:20:16.743852 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742115    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:20:16.743852 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742118    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:20:16.743852 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742120    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:20:16.743852 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742123    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:20:16.743852 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742125    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:20:16.743852 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742127    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:20:16.743852 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742130    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:20:16.743852 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742132    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:20:16.743852 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742135    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:20:16.743852 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742138    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:20:16.743852 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742141    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:20:16.743852 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742143    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:20:16.743852 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742146    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:20:16.743852 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742149    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:20:16.743852 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742152    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:20:16.743852 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742155    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:20:16.744354 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742159    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:20:16.744354 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742161    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:20:16.744354 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742164    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:20:16.744354 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742166    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:20:16.744354 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.742169    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:20:16.744354 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.743931    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:20:16.744354 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.743940    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:20:16.744354 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.743943    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:20:16.744354 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.743945    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:20:16.744354 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.743948    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:20:16.744354 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.743951    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:20:16.744354 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.743954    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:20:16.744354 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.743956    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:20:16.744354 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.743959    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:20:16.744354 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.743961    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:20:16.744354 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.743965    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:20:16.744354 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.743968    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:20:16.744354 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.743970    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:20:16.744354 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.743973    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:20:16.744354 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.743976    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:20:16.744964 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.743978    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:20:16.744964 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.743981    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:20:16.744964 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.743983    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:20:16.744964 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.743986    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:20:16.744964 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.743988    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:20:16.744964 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.743991    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:20:16.744964 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.743994    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:20:16.744964 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.743996    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:20:16.744964 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.743999    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:20:16.744964 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744002    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:20:16.744964 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744004    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:20:16.744964 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744006    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:20:16.744964 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744010    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:20:16.744964 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744013    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:20:16.744964 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744015    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:20:16.744964 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744018    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:20:16.744964 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744021    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:20:16.744964 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744024    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:20:16.744964 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744027    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:20:16.744964 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744029    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:20:16.745428 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744032    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:20:16.745428 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744034    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:20:16.745428 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744036    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:20:16.745428 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744039    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:20:16.745428 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744041    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:20:16.745428 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744044    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:20:16.745428 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744046    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:20:16.745428 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744049    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:20:16.745428 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744053    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:20:16.745428 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744057    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:20:16.745428 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744060    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:20:16.745428 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744063    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:20:16.745428 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744065    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:20:16.745428 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744068    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:20:16.745428 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744070    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:20:16.745428 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744073    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:20:16.745428 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744075    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:20:16.745428 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744078    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:20:16.745428 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744080    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:20:16.745897 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744083    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:20:16.745897 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744085    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:20:16.745897 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744088    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:20:16.745897 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744090    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:20:16.745897 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744092    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:20:16.745897 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744095    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:20:16.745897 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744098    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:20:16.745897 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744101    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:20:16.745897 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744104    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:20:16.745897 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744107    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:20:16.745897 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744109    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:20:16.745897 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744111    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:20:16.745897 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744114    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:20:16.745897 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744116    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:20:16.745897 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744119    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:20:16.745897 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744121    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:20:16.745897 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744123    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:20:16.745897 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744126    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:20:16.745897 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744128    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:20:16.745897 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744131    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:20:16.746398 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744141    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:20:16.746398 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744145    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:20:16.746398 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744149    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:20:16.746398 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744152    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:20:16.746398 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744155    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:20:16.746398 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744157    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:20:16.746398 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744160    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:20:16.746398 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744162    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:20:16.746398 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744165    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:20:16.746398 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744167    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:20:16.746398 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744169    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:20:16.746398 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744172    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:20:16.746398 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744174    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:20:16.746398 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744177    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:20:16.746398 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744179    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:20:16.746398 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744182    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:20:16.746398 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744186    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:20:16.746398 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744190    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:20:16.746398 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744193    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:20:16.746845 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744197    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:20:16.746845 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744200    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:20:16.746845 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744203    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:20:16.746845 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744205    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:20:16.746845 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744208    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:20:16.746845 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744210    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:20:16.746845 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744213    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:20:16.746845 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744215    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:20:16.746845 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744218    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:20:16.746845 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744222    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:20:16.746845 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744224    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:20:16.746845 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744227    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:20:16.746845 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744229    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:20:16.746845 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744739    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:20:16.746845 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744744    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:20:16.746845 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744747    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:20:16.746845 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744750    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:20:16.746845 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744753    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:20:16.746845 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744756    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:20:16.747310 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744759    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:20:16.747310 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744761    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:20:16.747310 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744764    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:20:16.747310 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744767    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:20:16.747310 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744770    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:20:16.747310 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744772    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:20:16.747310 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744775    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:20:16.747310 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744779    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:20:16.747310 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744782    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:20:16.747310 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744785    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:20:16.747310 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744787    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:20:16.747310 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744790    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:20:16.747310 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744793    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:20:16.747310 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744796    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:20:16.747310 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744799    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:20:16.747310 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744802    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:20:16.747310 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744804    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:20:16.747310 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744807    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:20:16.747310 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744810    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:20:16.747789 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744812    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:20:16.747789 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744815    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:20:16.747789 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744817    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:20:16.747789 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744819    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:20:16.747789 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744822    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:20:16.747789 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744824    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:20:16.747789 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744827    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:20:16.747789 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744829    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:20:16.747789 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744832    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:20:16.747789 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744834    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:20:16.747789 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744837    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:20:16.747789 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744840    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:20:16.747789 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744842    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:20:16.747789 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744844    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:20:16.747789 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744847    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:20:16.747789 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744849    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:20:16.747789 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744854    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:20:16.747789 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744880    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:20:16.747789 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744885    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:20:16.747789 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744889    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:20:16.748297 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744892    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:20:16.748297 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744894    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:20:16.748297 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744897    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:20:16.748297 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744900    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:20:16.748297 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744902    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:20:16.748297 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744905    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:20:16.748297 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744907    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:20:16.748297 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744910    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:20:16.748297 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744912    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:20:16.748297 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744915    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:20:16.748297 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744917    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:20:16.748297 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744920    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:20:16.748297 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744922    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:20:16.748297 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744925    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:20:16.748297 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744927    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:20:16.748297 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744930    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:20:16.748297 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744932    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:20:16.748297 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744935    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:20:16.748297 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744938    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:20:16.748297 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744940    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:20:16.748759 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744943    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:20:16.748759 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744946    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:20:16.748759 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744948    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:20:16.748759 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744951    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:20:16.748759 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744954    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:20:16.748759 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744957    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:20:16.748759 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744959    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:20:16.748759 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744962    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:20:16.748759 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744966    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:20:16.748759 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744968    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:20:16.748759 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744971    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:20:16.748759 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744974    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:20:16.748759 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744976    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:20:16.748759 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744979    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:20:16.748759 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744982    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:20:16.748759 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744984    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:20:16.748759 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744987    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:20:16.748759 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744989    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:20:16.748759 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744992    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:20:16.748759 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744994    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:20:16.749268 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.744997    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:20:16.749268 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745000    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:20:16.749268 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745003    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:20:16.749268 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745005    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:20:16.749268 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745008    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:20:16.749268 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745011    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:20:16.749268 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745013    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:20:16.749268 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745016    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:20:16.749268 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745018    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:20:16.749268 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745021    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:20:16.749268 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745023    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:20:16.749268 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745026    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:20:16.749268 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745028    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:20:16.749268 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745030    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:20:16.749268 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745033    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:20:16.749268 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745035    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:20:16.749268 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745038    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:20:16.749268 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745042    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:20:16.749268 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745045    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:20:16.749713 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745047    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:20:16.749713 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745050    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:20:16.749713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745146    2574 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 19:20:16.749713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745154    2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 19:20:16.749713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745161    2574 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 19:20:16.749713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745166    2574 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 19:20:16.749713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745170    2574 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 19:20:16.749713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745173    2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 19:20:16.749713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745177    2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 19:20:16.749713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745182    2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 19:20:16.749713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745185    2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 19:20:16.749713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745188    2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 19:20:16.749713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745191    2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 19:20:16.749713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745195    2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 19:20:16.749713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745198    2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 19:20:16.749713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745200    2574 flags.go:64] FLAG: --cgroup-root=""
Oct 30 19:20:16.749713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745204    2574 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 19:20:16.749713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745207    2574 flags.go:64] FLAG: --client-ca-file=""
Oct 30 19:20:16.749713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745210    2574 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 19:20:16.749713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745213    2574 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 19:20:16.749713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745217    2574 flags.go:64] FLAG: --cluster-domain=""
Oct 30 19:20:16.749713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745219    2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 19:20:16.749713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745222    2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 19:20:16.750270 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745225    2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 19:20:16.750270 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745229    2574 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 19:20:16.750270 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745237    2574 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 19:20:16.750270 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745240    2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 19:20:16.750270 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745243    2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 19:20:16.750270 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745246    2574 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 19:20:16.750270 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745249    2574 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 19:20:16.750270 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745253    2574 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 19:20:16.750270 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745256    2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 19:20:16.750270 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745259    2574 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 19:20:16.750270 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745262    2574 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 19:20:16.750270 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745267    2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 19:20:16.750270 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745270    2574 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 19:20:16.750270 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745273    2574 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 19:20:16.750270 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745276    2574 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 19:20:16.750270 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745279    2574 flags.go:64] FLAG: --enable-server="true"
Oct 30 19:20:16.750270 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745282    2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 19:20:16.750270 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745286    2574 flags.go:64] FLAG: --event-burst="100"
Oct 30 19:20:16.750270 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745289    2574 flags.go:64] FLAG: --event-qps="50"
Oct 30 19:20:16.750270 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745292    2574 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 19:20:16.750270 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745295    2574 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 19:20:16.750270 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745297    2574 flags.go:64] FLAG: --eviction-hard=""
Oct 30 19:20:16.750270 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745302    2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 19:20:16.750270 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745305    2574 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 19:20:16.750867 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745308    2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 19:20:16.750867 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745311    2574 flags.go:64] FLAG: --eviction-soft=""
Oct 30 19:20:16.750867 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745314    2574 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 19:20:16.750867 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745317    2574 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 19:20:16.750867 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745320    2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 19:20:16.750867 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745323    2574 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 19:20:16.750867 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745326    2574 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 19:20:16.750867 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745328    2574 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 19:20:16.750867 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745331    2574 flags.go:64] FLAG: --feature-gates=""
Oct 30 19:20:16.750867 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745335    2574 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 19:20:16.750867 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745338    2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 19:20:16.750867 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745342    2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 19:20:16.750867 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745345    2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 19:20:16.750867 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745347    2574 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 19:20:16.750867 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745350    2574 flags.go:64] FLAG: --help="false"
Oct 30 19:20:16.750867 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745354    2574 flags.go:64] FLAG: --hostname-override="ip-10-0-132-152.ec2.internal"
Oct 30 19:20:16.750867 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745362    2574 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 19:20:16.750867 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745365    2574 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 19:20:16.750867 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745368    2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 19:20:16.750867 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745372    2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 19:20:16.750867 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745375    2574 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 19:20:16.750867 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745378    2574 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 19:20:16.750867 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745381    2574 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 19:20:16.751410 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745384    2574 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 19:20:16.751410 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745386    2574 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 19:20:16.751410 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745389    2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 19:20:16.751410 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745392    2574 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 19:20:16.751410 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745395    2574 flags.go:64] FLAG: --kube-reserved=""
Oct 30 19:20:16.751410 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745398    2574 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 19:20:16.751410 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745401    2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 19:20:16.751410 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745404    2574 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 19:20:16.751410 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745407    2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 19:20:16.751410 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745410    2574 flags.go:64] FLAG: --lock-file=""
Oct 30 19:20:16.751410 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745412    2574 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 19:20:16.751410 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745415    2574 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 19:20:16.751410 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745418    2574 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 19:20:16.751410 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745424    2574 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 19:20:16.751410 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745427    2574 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 19:20:16.751410 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745430    2574 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 19:20:16.751410 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745433    2574 flags.go:64] FLAG: --logging-format="text"
Oct 30 19:20:16.751410 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745436    2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 19:20:16.751410 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745440    2574 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 19:20:16.751410 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745443    2574 flags.go:64] FLAG: --manifest-url=""
Oct 30 19:20:16.751410 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745446    2574 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 19:20:16.751410 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745451    2574 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 19:20:16.751410 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745454    2574 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 19:20:16.751410 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745458    2574 flags.go:64] FLAG: --max-pods="110"
Oct 30 19:20:16.751410 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745464    2574 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 19:20:16.752042 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745467    2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 19:20:16.752042 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745470    2574 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 19:20:16.752042 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745474    2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 19:20:16.752042 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745477    2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 19:20:16.752042 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745480    2574 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 19:20:16.752042 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745483    2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 19:20:16.752042 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745490    2574 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 19:20:16.752042 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745493    2574 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 19:20:16.752042 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745496    2574 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 19:20:16.752042 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745499    2574 flags.go:64] FLAG: --pod-cidr=""
Oct 30 19:20:16.752042 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745502    2574 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 19:20:16.752042 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745505    2574 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 19:20:16.752042 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745508    2574 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 19:20:16.752042 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745511    2574 flags.go:64] FLAG: --port="10250"
Oct 30 19:20:16.752042 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745514    2574 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 19:20:16.752042 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745516    2574 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-064ee3f159afc8f03"
Oct 30 19:20:16.752042 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745520    2574 flags.go:64] FLAG: --qos-reserved=""
Oct 30 19:20:16.752042 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745523    2574 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 19:20:16.752042 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745526    2574 flags.go:64] FLAG: --register-node="true"
Oct 30 19:20:16.752042 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745529    2574 flags.go:64] FLAG: --register-with-taints=""
Oct 30 19:20:16.752042 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745533    2574 flags.go:64] FLAG: --registry-burst="10"
Oct 30 19:20:16.752042 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745535    2574 flags.go:64] FLAG: --registry-qps="5"
Oct 30 19:20:16.752042 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745538    2574 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 19:20:16.752042 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745541    2574 flags.go:64] FLAG: --reserved-memory=""
Oct 30 19:20:16.752042 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745545    2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 19:20:16.752042 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745548    2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 19:20:16.752713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745551    2574 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 19:20:16.752713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745554    2574 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 19:20:16.752713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745557    2574 flags.go:64] FLAG: --runonce="false"
Oct 30 19:20:16.752713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745560    2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 19:20:16.752713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745566    2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 19:20:16.752713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745570    2574 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 19:20:16.752713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745573    2574 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 19:20:16.752713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745576    2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 19:20:16.752713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745581    2574 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 19:20:16.752713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745585    2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 19:20:16.752713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745588    2574 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 19:20:16.752713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745590    2574 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 19:20:16.752713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745593    2574 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 19:20:16.752713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745596    2574 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 19:20:16.752713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745599    2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 19:20:16.752713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745602    2574 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 19:20:16.752713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745605    2574 flags.go:64] FLAG: --system-cgroups=""
Oct 30 19:20:16.752713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745608    2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 19:20:16.752713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745614    2574 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 19:20:16.752713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745617    2574 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 19:20:16.752713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745619    2574 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 19:20:16.752713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745623    2574 flags.go:64] FLAG: --tls-min-version=""
Oct 30 19:20:16.752713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745626    2574 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 19:20:16.752713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745629    2574 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 19:20:16.752713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745632    2574 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 19:20:16.753325 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745634    2574 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 19:20:16.753325 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745637    2574 flags.go:64] FLAG: --v="2"
Oct 30 19:20:16.753325 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745641    2574 flags.go:64] FLAG: --version="false"
Oct 30 19:20:16.753325 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745645    2574 flags.go:64] FLAG: --vmodule=""
Oct 30 19:20:16.753325 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745649    2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 19:20:16.753325 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.745653    2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 19:20:16.753325 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745794    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:20:16.753325 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745798    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:20:16.753325 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745802    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:20:16.753325 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745805    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:20:16.753325 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745808    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:20:16.753325 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745810    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:20:16.753325 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745814    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:20:16.753325 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745817    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:20:16.753325 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745820    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:20:16.753325 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745824    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:20:16.753325 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745827    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:20:16.753325 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745830    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:20:16.753325 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745832    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:20:16.753325 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745835    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:20:16.753325 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745838    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:20:16.753822 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745840    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:20:16.753822 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745843    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:20:16.753822 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745845    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:20:16.753822 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745847    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:20:16.753822 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745850    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:20:16.753822 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745852    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:20:16.753822 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745855    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:20:16.753822 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745875    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:20:16.753822 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745880    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:20:16.753822 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745883    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:20:16.753822 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745886    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:20:16.753822 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745888    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:20:16.753822 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745891    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:20:16.753822 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745894    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:20:16.753822 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745896    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:20:16.753822 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745899    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:20:16.753822 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745902    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:20:16.753822 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745904    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:20:16.753822 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745907    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:20:16.754284 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745910    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:20:16.754284 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745912    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:20:16.754284 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745915    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:20:16.754284 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745917    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:20:16.754284 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745921    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:20:16.754284 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745923    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:20:16.754284 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745926    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:20:16.754284 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745930    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:20:16.754284 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745933    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:20:16.754284 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745937    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:20:16.754284 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745939    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:20:16.754284 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745942    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:20:16.754284 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745946    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:20:16.754284 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745949    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:20:16.754284 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745953    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:20:16.754284 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745956    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:20:16.754284 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745960    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:20:16.754284 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745962    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:20:16.754284 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745965    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:20:16.754734 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745967    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:20:16.754734 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745970    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:20:16.754734 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745972    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:20:16.754734 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745975    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:20:16.754734 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745977    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:20:16.754734 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745980    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:20:16.754734 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745982    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:20:16.754734 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745984    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:20:16.754734 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745987    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:20:16.754734 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745989    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:20:16.754734 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745992    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:20:16.754734 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745994    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:20:16.754734 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745996    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:20:16.754734 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.745999    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:20:16.754734 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746001    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:20:16.754734 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746004    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:20:16.754734 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746007    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:20:16.754734 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746010    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:20:16.754734 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746013    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:20:16.754734 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746016    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:20:16.755210 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746019    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:20:16.755210 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746022    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:20:16.755210 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746024    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:20:16.755210 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746028    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:20:16.755210 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746030    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:20:16.755210 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746033    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:20:16.755210 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746035    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:20:16.755210 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746038    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:20:16.755210 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746040    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:20:16.755210 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746043    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:20:16.755210 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746045    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:20:16.755210 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746048    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:20:16.755210 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746050    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:20:16.755210 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746053    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:20:16.755210 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746055    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:20:16.755210 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746058    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:20:16.755210 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746061    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:20:16.755210 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746066    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:20:16.755210 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746069    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:20:16.755660 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746071    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:20:16.755660 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746074    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:20:16.755660 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746076    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:20:16.755660 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746079    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:20:16.755660 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746081    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:20:16.755660 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746084    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:20:16.755660 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746086    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:20:16.755660 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746089    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:20:16.755660 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746091    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:20:16.755660 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746094    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:20:16.755660 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746098    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:20:16.755660 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746100    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:20:16.755660 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746103    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:20:16.755660 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746106    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:20:16.755660 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.746133    2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 19:20:16.755660 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746178    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:20:16.756051 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746183    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:20:16.756051 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746187    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:20:16.756051 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746190    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:20:16.756051 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746193    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:20:16.756051 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746196    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:20:16.756051 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746199    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:20:16.756051 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746202    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:20:16.756051 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746204    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:20:16.756051 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746207    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:20:16.756051 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746209    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:20:16.756051 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746212    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:20:16.756051 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746214    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:20:16.756051 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746217    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:20:16.756051 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746219    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:20:16.756051 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746222    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:20:16.756051 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746224    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:20:16.756051 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746227    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:20:16.756051 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746229    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:20:16.756051 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746232    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:20:16.756493 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746234    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:20:16.756493 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746237    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:20:16.756493 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746240    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:20:16.756493 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746244    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:20:16.756493 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746247    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:20:16.756493 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746251    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:20:16.756493 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746254    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:20:16.756493 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746256    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:20:16.756493 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746260    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:20:16.756493 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746263    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:20:16.756493 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746266    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:20:16.756493 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746269    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:20:16.756493 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746272    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:20:16.756493 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746274    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:20:16.756493 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746277    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:20:16.756493 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746280    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:20:16.756493 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746283    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:20:16.756493 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746285    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:20:16.756493 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746288    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:20:16.756493 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746290    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:20:16.757000 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746292    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:20:16.757000 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746295    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:20:16.757000 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746297    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:20:16.757000 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746300    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:20:16.757000 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746302    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:20:16.757000 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746305    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:20:16.757000 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746307    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:20:16.757000 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746310    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:20:16.757000 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746312    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:20:16.757000 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746314    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:20:16.757000 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746317    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:20:16.757000 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746319    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:20:16.757000 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746322    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:20:16.757000 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746325    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:20:16.757000 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746327    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:20:16.757000 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746330    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:20:16.757000 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746332    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:20:16.757000 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746334    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:20:16.757000 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746337    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:20:16.757000 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746339    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:20:16.757475 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746343    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:20:16.757475 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746346    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:20:16.757475 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746350    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:20:16.757475 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746352    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:20:16.757475 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746355    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:20:16.757475 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746359    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:20:16.757475 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746362    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:20:16.757475 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746365    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:20:16.757475 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746368    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:20:16.757475 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746371    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:20:16.757475 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746373    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:20:16.757475 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746376    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:20:16.757475 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746378    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:20:16.757475 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746381    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:20:16.757475 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746383    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:20:16.757475 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746386    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:20:16.757475 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746388    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:20:16.757475 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746391    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:20:16.757475 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746393    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:20:16.758213 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746396    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:20:16.758213 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746398    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:20:16.758213 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746401    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:20:16.758213 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746404    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:20:16.758213 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746406    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:20:16.758213 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746409    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:20:16.758213 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746412    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:20:16.758213 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746414    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:20:16.758213 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746417    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:20:16.758213 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746419    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:20:16.758213 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746422    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:20:16.758213 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746424    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:20:16.758213 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746427    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:20:16.758213 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746430    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:20:16.758213 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746433    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:20:16.758213 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746437    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:20:16.758213 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746439    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:20:16.758213 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746442    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:20:16.758213 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746444    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:20:16.758659 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746447    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:20:16.758659 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746449    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:20:16.758659 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746452    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:20:16.758659 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746455    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:20:16.758659 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746458    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:20:16.758659 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746460    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:20:16.758659 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746463    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:20:16.758659 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.746466    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:20:16.758659 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.753442    2574 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 19:20:16.758659 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.753460    2574 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 19:20:16.758659 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.753475    2574 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 19:20:16.758659 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.753480    2574 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 19:20:16.758659 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753537    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:20:16.758659 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753542    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:20:16.758659 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753545    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:20:16.758659 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753548    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:20:16.758659 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753551    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:20:16.758659 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753553    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:20:16.758659 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753556    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:20:16.759412 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753559    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:20:16.759412 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753561    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:20:16.759412 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753564    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:20:16.759412 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753566    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:20:16.759412 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753569    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:20:16.759412 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753571    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:20:16.759412 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753574    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:20:16.759412 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753577    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:20:16.759412 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753579    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:20:16.759412 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753581    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:20:16.759412 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753584    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:20:16.759412 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753586    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:20:16.759412 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753589    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:20:16.759412 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753591    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:20:16.759412 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753593    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:20:16.759412 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753596    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:20:16.759412 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753598    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:20:16.759412 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753600    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:20:16.759412 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753603    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:20:16.759412 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753605    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:20:16.760097 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753608    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:20:16.760097 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753611    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:20:16.760097 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753615    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:20:16.760097 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753618    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:20:16.760097 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753621    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:20:16.760097 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753623    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:20:16.760097 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753626    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:20:16.760097 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753629    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:20:16.760097 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753631    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:20:16.760097 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753634    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:20:16.760097 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753636    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:20:16.760097 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753639    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:20:16.760097 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753641    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:20:16.760097 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753644    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:20:16.760097 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753646    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:20:16.760097 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753649    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:20:16.760097 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753651    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:20:16.760097 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753653    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:20:16.760097 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753656    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:20:16.760097 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753658    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:20:16.760878 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753661    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:20:16.760878 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753663    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:20:16.760878 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753666    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:20:16.760878 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753668    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:20:16.760878 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753671    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:20:16.760878 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753673    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:20:16.760878 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753676    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:20:16.760878 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753678    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:20:16.760878 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753680    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:20:16.760878 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753683    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:20:16.760878 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753685    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:20:16.760878 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753688    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:20:16.760878 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753690    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:20:16.760878 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753694    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:20:16.760878 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753696    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:20:16.760878 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753706    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:20:16.760878 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753709    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:20:16.760878 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753712    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:20:16.760878 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753714    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:20:16.760878 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753717    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:20:16.761537 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753719    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:20:16.761537 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753722    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:20:16.761537 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753726    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:20:16.761537 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753730    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:20:16.761537 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753733    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:20:16.761537 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753735    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:20:16.761537 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753738    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:20:16.761537 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753740    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:20:16.761537 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753742    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:20:16.761537 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753745    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:20:16.761537 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753747    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:20:16.761537 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753750    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:20:16.761537 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753752    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:20:16.761537 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753755    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:20:16.761537 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753757    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:20:16.761537 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753760    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:20:16.761537 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753762    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:20:16.761537 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753765    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:20:16.761537 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753767    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:20:16.762019 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753770    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:20:16.762019 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753772    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:20:16.762019 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753775    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:20:16.762019 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753777    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:20:16.762019 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753780    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:20:16.762019 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753782    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:20:16.762019 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753786    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:20:16.762019 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753789    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:20:16.762019 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753793    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:20:16.762019 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753796    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:20:16.762019 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753799    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:20:16.762019 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753801    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:20:16.762019 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753803    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:20:16.762019 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753806    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:20:16.762019 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753808    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:20:16.762019 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753811    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:20:16.762019 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753813    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:20:16.762019 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753816    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:20:16.762019 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753818    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:20:16.762547 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.753821    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:20:16.762547 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.753850    2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 19:20:16.762547 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754033    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:20:16.762547 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754038    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:20:16.762547 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754042    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:20:16.762547 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754044    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:20:16.762547 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754048    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:20:16.762547 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754050    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:20:16.762547 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754053    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:20:16.762547 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754055    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:20:16.762547 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754058    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:20:16.762547 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754061    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:20:16.762547 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754065    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:20:16.762547 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754068    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:20:16.762547 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754071    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:20:16.762981 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754074    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:20:16.762981 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754077    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:20:16.762981 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754080    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:20:16.762981 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754082    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:20:16.762981 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754085    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:20:16.762981 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754088    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:20:16.762981 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754091    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:20:16.762981 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754094    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:20:16.762981 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754097    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:20:16.762981 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754100    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:20:16.762981 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754102    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:20:16.762981 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754105    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:20:16.762981 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754107    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:20:16.762981 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754110    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:20:16.762981 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754112    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:20:16.762981 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754115    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:20:16.762981 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754118    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:20:16.762981 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754120    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:20:16.762981 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754123    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:20:16.762981 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754125    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:20:16.763657 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754128    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:20:16.763657 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754130    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:20:16.763657 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754133    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:20:16.763657 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754135    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:20:16.763657 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754137    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:20:16.763657 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754140    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:20:16.763657 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754142    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:20:16.763657 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754145    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:20:16.763657 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754147    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:20:16.763657 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754150    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:20:16.763657 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754152    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:20:16.763657 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754154    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:20:16.763657 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754157    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:20:16.763657 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754159    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:20:16.763657 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754162    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:20:16.763657 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754164    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:20:16.763657 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754167    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:20:16.763657 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754170    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:20:16.763657 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754173    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:20:16.763657 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754175    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:20:16.764267 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754178    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:20:16.764267 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754181    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:20:16.764267 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754183    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:20:16.764267 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754186    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:20:16.764267 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754188    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:20:16.764267 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754190    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:20:16.764267 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754193    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:20:16.764267 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754195    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:20:16.764267 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754197    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:20:16.764267 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754200    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:20:16.764267 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754202    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:20:16.764267 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754205    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:20:16.764267 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754207    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:20:16.764267 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754209    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:20:16.764267 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754212    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:20:16.764267 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754214    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:20:16.764267 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754216    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:20:16.764267 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754219    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:20:16.764267 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754221    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:20:16.764267 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754224    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:20:16.764835 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754226    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:20:16.764835 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754229    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:20:16.764835 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754231    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:20:16.764835 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754233    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:20:16.764835 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754236    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:20:16.764835 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754238    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:20:16.764835 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754241    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:20:16.764835 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754243    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:20:16.764835 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754246    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:20:16.764835 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754248    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:20:16.764835 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754251    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:20:16.764835 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754253    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:20:16.764835 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754256    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:20:16.764835 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754259    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:20:16.764835 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754264    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:20:16.764835 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754267    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:20:16.764835 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754270    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:20:16.764835 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754273    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:20:16.764835 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754276    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:20:16.765302 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754278    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:20:16.765302 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754281    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:20:16.765302 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754283    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:20:16.765302 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754286    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:20:16.765302 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754288    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:20:16.765302 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754291    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:20:16.765302 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754293    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:20:16.765302 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754296    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:20:16.765302 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754299    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:20:16.765302 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754301    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:20:16.765302 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754304    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:20:16.765302 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754306    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:20:16.765302 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754309    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:20:16.765302 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754312    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:20:16.765302 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.754334    2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 19:20:16.765302 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754367    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:20:16.765697 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754371    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:20:16.765697 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754375    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:20:16.765697 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754378    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:20:16.765697 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754381    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:20:16.765697 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754384    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:20:16.765697 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754386    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:20:16.765697 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754389    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:20:16.765697 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754392    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:20:16.765697 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754394    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:20:16.765697 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754397    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:20:16.765697 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754399    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:20:16.765697 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754402    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:20:16.765697 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754405    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:20:16.765697 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754408    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:20:16.765697 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754410    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:20:16.765697 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754413    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:20:16.765697 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754415    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:20:16.765697 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754417    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:20:16.765697 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754420    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:20:16.765697 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754422    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:20:16.766196 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754425    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:20:16.766196 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754428    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:20:16.766196 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754431    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:20:16.766196 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754434    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:20:16.766196 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754436    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:20:16.766196 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754439    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:20:16.766196 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754442    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:20:16.766196 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754444    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:20:16.766196 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754447    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:20:16.766196 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754449    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:20:16.766196 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754452    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:20:16.766196 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754454    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:20:16.766196 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754456    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:20:16.766196 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754459    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:20:16.766196 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754461    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:20:16.766196 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754464    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:20:16.766196 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754466    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:20:16.766196 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754468    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:20:16.766196 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754471    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:20:16.766629 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754473    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:20:16.766629 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754476    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:20:16.766629 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754478    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:20:16.766629 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754480    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:20:16.766629 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754483    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:20:16.766629 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754486    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:20:16.766629 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754489    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:20:16.766629 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754492    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:20:16.766629 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754495    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:20:16.766629 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754497    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:20:16.766629 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754499    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:20:16.766629 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754503    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:20:16.766629 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754507    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:20:16.766629 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754510    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:20:16.766629 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754513    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:20:16.766629 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754516    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:20:16.766629 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754518    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:20:16.766629 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754520    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:20:16.766629 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754523    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:20:16.767087 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754525    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:20:16.767087 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754527    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:20:16.767087 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754534    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:20:16.767087 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754536    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:20:16.767087 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754539    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:20:16.767087 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754541    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:20:16.767087 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754544    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:20:16.767087 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754546    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:20:16.767087 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754549    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:20:16.767087 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754551    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:20:16.767087 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754553    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:20:16.767087 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754556    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:20:16.767087 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754559    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:20:16.767087 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754561    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:20:16.767087 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754563    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:20:16.767087 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754566    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:20:16.767087 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754568    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:20:16.767087 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754571    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:20:16.767087 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754573    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:20:16.767087 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754576    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:20:16.767791 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754578    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:20:16.767791 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754581    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:20:16.767791 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754584    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:20:16.767791 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754586    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:20:16.767791 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754589    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:20:16.767791 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754591    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:20:16.767791 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754593    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:20:16.767791 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754596    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:20:16.767791 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754598    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:20:16.767791 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754601    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:20:16.767791 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754603    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:20:16.767791 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754605    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:20:16.767791 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754609    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:20:16.767791 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754611    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:20:16.767791 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754615    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:20:16.767791 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754618    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:20:16.767791 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754620    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:20:16.767791 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754622    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:20:16.767791 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754625    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:20:16.768320 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754628    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:20:16.768320 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754630    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:20:16.768320 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754633    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:20:16.768320 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754635    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:20:16.768320 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754637    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:20:16.768320 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754640    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:20:16.768320 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754642    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:20:16.768320 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:16.754645    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:20:16.768320 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.754702    2574 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 19:20:16.768320 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.756629    2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 19:20:16.768320 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.757723    2574 server.go:1014] "Starting client certificate rotation"
Oct 30 19:20:16.768320 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.757844    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 19:20:16.768320 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.757909    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 19:20:16.768320 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.759564    2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 19:20:16.768320 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.764555    2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 19:20:16.775887 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.775851    2574 log.go:25] "Validated CRI v1 runtime API"
Oct 30 19:20:16.779198 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.779182    2574 log.go:25] "Validated CRI v1 image API"
Oct 30 19:20:16.780074 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.780046    2574 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 19:20:16.785104 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.785069    2574 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8d6b887f-5690-45d6-972e-64046250ab11:/dev/nvme0n1p4 f5ae08bf-ff2d-4ff4-b874-94da9c7dd4dd:/dev/nvme0n1p3]
Oct 30 19:20:16.785104 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.785097    2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 19:20:16.789340 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.789224    2574 manager.go:217] Machine: {Timestamp:2026-10-30 19:20:16.788741895 +0000 UTC m=+0.344964162 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102423 MemoryCapacity:32812015616 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec268f2c3424d8ca562362417016b90e SystemUUID:ec268f2c-3424-d8ca-5623-62417016b90e BootID:c43b134f-0af3-4ca4-adbe-6b35b4b990c9 Filesystems:[{Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406007808 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:d4:cf:fe:c1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:d4:cf:fe:c1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6a:8a:67:d1:90:ba Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812015616 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 19:20:16.789340 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.789338    2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 19:20:16.789472 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.789456    2574 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 19:20:16.792229 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.792201    2574 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 19:20:16.792373 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.792230    2574 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-152.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 19:20:16.793135 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.793125    2574 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 19:20:16.793169 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.793136    2574 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 19:20:16.793169 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.793155    2574 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 19:20:16.793263 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.793168    2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 19:20:16.793966 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.793956    2574 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 19:20:16.797682 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.797669    2574 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 19:20:16.797812 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.797802    2574 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 19:20:16.798501 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.798490    2574 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 19:20:16.798557 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.798509    2574 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 19:20:16.798557 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.798532    2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 19:20:16.798557 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.798546    2574 kubelet.go:405] "Adding apiserver pod source"
Oct 30 19:20:16.798557 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.798555    2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 19:20:16.799031 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.799012    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 19:20:16.801206 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.801189    2574 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 19:20:16.807343 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.807323    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 19:20:16.808284 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.808265    2574 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 19:20:16.808353 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.808314    2574 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 19:20:16.808529 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.808517    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 19:20:16.808567 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.808539    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 19:20:16.808567 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.808549    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 19:20:16.808567 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.808558    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 19:20:16.808567 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.808566    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 19:20:16.808673 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.808575    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 19:20:16.809291 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.809110    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 19:20:16.809291 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.809119    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 19:20:16.809291 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.809126    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 19:20:16.809291 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.809133    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 19:20:16.809291 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.809141    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 19:20:16.809291 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.809150    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 19:20:16.810032 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.810022    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 19:20:16.810032 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.810033    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 19:20:16.813839 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.813825    2574 server.go:1263] "Started kubelet"
Oct 30 19:20:16.813956 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.813922    2574 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 19:20:16.813992 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.813933    2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 19:20:16.814029 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.814006    2574 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 19:20:16.814528 ip-10-0-132-152 systemd[1]: Started Kubernetes Kubelet.
Oct 30 19:20:16.815208 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.815190    2574 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 19:20:16.816382 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.816364    2574 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 19:20:16.819318 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:16.819299    2574 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 19:20:16.820424 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.820408    2574 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 19:20:16.820533 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.820511    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 19:20:16.820533 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:16.820517    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-152.ec2.internal\" not found"
Oct 30 19:20:16.821019 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.820988    2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 19:20:16.821019 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.820991    2574 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 19:20:16.821148 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.821029    2574 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 19:20:16.821216 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.821200    2574 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 19:20:16.821216 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.821212    2574 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 19:20:16.823553 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.823536    2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 19:20:16.823553 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.823553    2574 factory.go:55] Registering systemd factory
Oct 30 19:20:16.823690 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.823561    2574 factory.go:223] Registration of the systemd container factory successfully
Oct 30 19:20:16.823831 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.823813    2574 factory.go:153] Registering CRI-O factory
Oct 30 19:20:16.823918 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.823837    2574 factory.go:223] Registration of the crio container factory successfully
Oct 30 19:20:16.823918 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.823875    2574 factory.go:103] Registering Raw factory
Oct 30 19:20:16.823918 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.823895    2574 manager.go:1199] Started watching for new ooms in manager
Oct 30 19:20:16.824929 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.824914    2574 manager.go:319] Starting recovery of all containers
Oct 30 19:20:16.837145 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.837119    2574 manager.go:324] Recovery completed
Oct 30 19:20:16.841174 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.841154    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 19:20:16.843146 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.843114    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-152.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 19:20:16.843218 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.843164    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-152.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 19:20:16.843218 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.843180    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-152.ec2.internal" event="NodeHasSufficientPID"
Oct 30 19:20:16.843376 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.843350    2574 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 19:20:16.843905 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.843892    2574 cpu_manager.go:226] "Starting" policy="none"
Oct 30 19:20:16.843905 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.843903    2574 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 19:20:16.844035 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.843898    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 19:20:16.844035 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.843919    2574 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 19:20:16.844714 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.844699    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-152.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 19:20:16.844757 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.844727    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-152.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 19:20:16.844757 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.844736    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-152.ec2.internal" event="NodeHasSufficientPID"
Oct 30 19:20:16.846134 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.846122    2574 policy_none.go:50] "Start"
Oct 30 19:20:16.846178 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.846137    2574 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 19:20:16.846178 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.846147    2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 19:20:16.846231 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.846197    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6w28r"
Oct 30 19:20:16.848118 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.848108    2574 policy_none.go:44] "Start"
Oct 30 19:20:16.854069 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.853939    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6w28r"
Oct 30 19:20:16.855991 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:16.854927    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-152.ec2.internal.18e364e646a0de5d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-152.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-152.ec2.internal,},FirstTimestamp:2026-10-30 19:20:16.813792861 +0000 UTC m=+0.370015130,LastTimestamp:2026-10-30 19:20:16.813792861 +0000 UTC m=+0.370015130,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-152.ec2.internal,}"
Oct 30 19:20:16.868442 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:16.868419    2574 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-152.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 19:20:16.868535 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.868465    2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-152.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 19:20:16.869950 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:16.869919    2574 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Oct 30 19:20:16.870037 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:16.870005    2574 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-152.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 30 19:20:16.870103 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:16.869995    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-152.ec2.internal.18e364e64860be83  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-152.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-152.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-152.ec2.internal,},FirstTimestamp:2026-10-30 19:20:16.843144835 +0000 UTC m=+0.399367103,LastTimestamp:2026-10-30 19:20:16.843144835 +0000 UTC m=+0.399367103,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-152.ec2.internal,}"
Oct 30 19:20:16.882648 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:16.882621    2574 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 30 19:20:16.890538 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.890500    2574 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 19:20:16.893226 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.893209    2574 manager.go:341] "Starting Device Plugin manager"
Oct 30 19:20:16.893336 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:16.893260    2574 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 19:20:16.893336 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.893272    2574 server.go:87] "Starting device plugin registration server"
Oct 30 19:20:16.893574 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.893561    2574 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 19:20:16.893636 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.893576    2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 19:20:16.893682 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.893670    2574 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 19:20:16.893766 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.893748    2574 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 19:20:16.893766 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.893762    2574 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 19:20:16.894445 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:16.894428    2574 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 19:20:16.894533 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:16.894469    2574 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-152.ec2.internal\" not found"
Oct 30 19:20:16.947021 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.946987    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 19:20:16.949133 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.948330    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 19:20:16.949133 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.948348    2574 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 19:20:16.949133 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.948368    2574 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 19:20:16.949133 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:16.948406    2574 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 19:20:16.966255 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.966237    2574 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:20:16.995324 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.995256    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 19:20:16.996283 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.996266    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-152.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 19:20:16.996389 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.996301    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-152.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 19:20:16.996389 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.996316    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-152.ec2.internal" event="NodeHasSufficientPID"
Oct 30 19:20:16.996389 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:16.996346    2574 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-132-152.ec2.internal"
Oct 30 19:20:17.016307 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.016282    2574 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-132-152.ec2.internal"
Oct 30 19:20:17.016436 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:17.016313    2574 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-152.ec2.internal\": node \"ip-10-0-132-152.ec2.internal\" not found"
Oct 30 19:20:17.049281 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.049243    2574 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-152.ec2.internal"]
Oct 30 19:20:17.049426 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.049312    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 19:20:17.050219 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.050205    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-152.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 19:20:17.050295 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.050235    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-152.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 19:20:17.050295 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.050253    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-152.ec2.internal" event="NodeHasSufficientPID"
Oct 30 19:20:17.052431 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.052416    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 19:20:17.052583 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.052566    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal"
Oct 30 19:20:17.052625 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.052600    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 19:20:17.053220 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.053202    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-152.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 19:20:17.053312 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.053243    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-152.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 19:20:17.053312 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.053242    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-152.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 19:20:17.053312 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.053259    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-152.ec2.internal" event="NodeHasSufficientPID"
Oct 30 19:20:17.053312 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.053267    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-152.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 19:20:17.053312 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.053278    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-152.ec2.internal" event="NodeHasSufficientPID"
Oct 30 19:20:17.055366 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.055351    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-152.ec2.internal"
Oct 30 19:20:17.055436 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.055384    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 19:20:17.056425 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.056405    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-152.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 19:20:17.056517 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.056441    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-152.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 19:20:17.056517 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.056456    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-152.ec2.internal" event="NodeHasSufficientPID"
Oct 30 19:20:17.072027 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:17.072000    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-152.ec2.internal\" not found"
Oct 30 19:20:17.082740 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:17.082715    2574 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-152.ec2.internal\" not found" node="ip-10-0-132-152.ec2.internal"
Oct 30 19:20:17.085652 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:17.085633    2574 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-152.ec2.internal\" not found" node="ip-10-0-132-152.ec2.internal"
Oct 30 19:20:17.122987 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.122953    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3412174e80de20d346c1e09c794a54c8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal\" (UID: \"3412174e80de20d346c1e09c794a54c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal"
Oct 30 19:20:17.122987 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.122991    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3412174e80de20d346c1e09c794a54c8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal\" (UID: \"3412174e80de20d346c1e09c794a54c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal"
Oct 30 19:20:17.123171 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.123039    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e58ff4b1131d98fcba47fc4e4f2a3553-config\") pod \"kube-apiserver-proxy-ip-10-0-132-152.ec2.internal\" (UID: \"e58ff4b1131d98fcba47fc4e4f2a3553\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-152.ec2.internal"
Oct 30 19:20:17.172770 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:17.172737    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-152.ec2.internal\" not found"
Oct 30 19:20:17.224098 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.224060    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3412174e80de20d346c1e09c794a54c8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal\" (UID: \"3412174e80de20d346c1e09c794a54c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal"
Oct 30 19:20:17.224098 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.224099    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3412174e80de20d346c1e09c794a54c8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal\" (UID: \"3412174e80de20d346c1e09c794a54c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal"
Oct 30 19:20:17.224323 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.224125    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e58ff4b1131d98fcba47fc4e4f2a3553-config\") pod \"kube-apiserver-proxy-ip-10-0-132-152.ec2.internal\" (UID: \"e58ff4b1131d98fcba47fc4e4f2a3553\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-152.ec2.internal"
Oct 30 19:20:17.224323 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.224194    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e58ff4b1131d98fcba47fc4e4f2a3553-config\") pod \"kube-apiserver-proxy-ip-10-0-132-152.ec2.internal\" (UID: \"e58ff4b1131d98fcba47fc4e4f2a3553\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-152.ec2.internal"
Oct 30 19:20:17.224323 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.224198    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3412174e80de20d346c1e09c794a54c8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal\" (UID: \"3412174e80de20d346c1e09c794a54c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal"
Oct 30 19:20:17.224323 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.224198    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3412174e80de20d346c1e09c794a54c8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal\" (UID: \"3412174e80de20d346c1e09c794a54c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal"
Oct 30 19:20:17.273587 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:17.273524    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-152.ec2.internal\" not found"
Oct 30 19:20:17.374368 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:17.374335    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-152.ec2.internal\" not found"
Oct 30 19:20:17.386549 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.386519    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal"
Oct 30 19:20:17.388605 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.388588    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-152.ec2.internal"
Oct 30 19:20:17.475461 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:17.475418    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-152.ec2.internal\" not found"
Oct 30 19:20:17.575969 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:17.575886    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-152.ec2.internal\" not found"
Oct 30 19:20:17.676377 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:17.676339    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-152.ec2.internal\" not found"
Oct 30 19:20:17.757874 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.757823    2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 19:20:17.758342 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.758012    2574 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 19:20:17.777134 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:17.777103    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-152.ec2.internal\" not found"
Oct 30 19:20:17.858844 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.858788    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 19:15:17 +0000 UTC" deadline="2028-03-30 11:20:29.507792533 +0000 UTC"
Oct 30 19:20:17.858844 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.858839    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12400h0m11.648956041s"
Oct 30 19:20:17.877716 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:17.877672    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-152.ec2.internal\" not found"
Oct 30 19:20:17.896173 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.896125    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 19:20:17.897688 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.897664    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-152.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 19:20:17.897688 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.897700    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-152.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 19:20:17.897901 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.897711    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-152.ec2.internal" event="NodeHasSufficientPID"
Oct 30 19:20:17.977933 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:17.977906    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-152.ec2.internal\" not found"
Oct 30 19:20:17.982430 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.982385    2574 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:20:17.985148 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:17.985124    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode58ff4b1131d98fcba47fc4e4f2a3553.slice/crio-6ac107b4bbb8ab01e09738c400166c19b041992f42a2ea5a87e53022c6317bd2 WatchSource:0}: Error finding container 6ac107b4bbb8ab01e09738c400166c19b041992f42a2ea5a87e53022c6317bd2: Status 404 returned error can't find the container with id 6ac107b4bbb8ab01e09738c400166c19b041992f42a2ea5a87e53022c6317bd2
Oct 30 19:20:17.985707 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:17.985693    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3412174e80de20d346c1e09c794a54c8.slice/crio-fe16d935efdb60f47b80f91c0cd9440e9da8c1aa652bdf06a543d6ccec9b3708 WatchSource:0}: Error finding container fe16d935efdb60f47b80f91c0cd9440e9da8c1aa652bdf06a543d6ccec9b3708: Status 404 returned error can't find the container with id fe16d935efdb60f47b80f91c0cd9440e9da8c1aa652bdf06a543d6ccec9b3708
Oct 30 19:20:17.990437 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:17.990417    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:20:18.021396 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.021354    2574 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal"
Oct 30 19:20:18.035250 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.035225    2574 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:20:18.035341 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.035313    2574 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:20:18.044116 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.044098    2574 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 19:20:18.045236 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.045223    2574 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-152.ec2.internal"
Oct 30 19:20:18.066194 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.066175    2574 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 19:20:18.799359 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.799326    2574 apiserver.go:52] "Watching apiserver"
Oct 30 19:20:18.821731 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.821698    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 19:20:18.839936 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.839906    2574 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 19:20:18.840356 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.840325    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-c9vlh","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch","openshift-dns/node-resolver-f7tbh","openshift-network-diagnostics/network-check-target-5j799","kube-system/konnectivity-agent-nw4jx","kube-system/kube-apiserver-proxy-ip-10-0-132-152.ec2.internal","openshift-ovn-kubernetes/ovnkube-node-8pv94","kube-system/global-pull-secret-syncer-z7mhx","openshift-cluster-node-tuning-operator/tuned-dvzwm","openshift-image-registry/node-ca-tgt5g","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal","openshift-multus/multus-sdlcz","openshift-multus/multus-additional-cni-plugins-hcmmq","openshift-multus/network-metrics-daemon-9jk99"]
Oct 30 19:20:18.843395 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.843368    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:18.845548 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.845524    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:20:18.848245 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.847955    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-f7tbh"
Oct 30 19:20:18.850785 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.850762    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:20:18.850916 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:18.850867    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5j799" podUID="cd444ba1-edf1-49aa-a0cf-b863c93e60be"
Oct 30 19:20:18.853119 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.853068    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nw4jx"
Oct 30 19:20:18.858976 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.858953    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8rhbj\""
Oct 30 19:20:18.859165 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.859152    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 19:20:18.859395 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.859380    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 19:20:18.859632 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.859610    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 19:20:18.860530 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.859873    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 19:20:18.860530 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.860026    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bkrzr\""
Oct 30 19:20:18.860530 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.860140    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 19:20:18.860530 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.860416    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 19:20:18.860530 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.860458    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 19:20:18.860530 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.860493    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 19:20:18.860898 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.860713    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 19:20:18.862227 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.862204    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:20:18.862338 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:18.862296    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jk99" podUID="41ff5ee2-70a2-4baf-af4e-00e6575c2e99"
Oct 30 19:20:18.866080 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.865908    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:18.867919 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.867894    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-c9vlh"
Oct 30 19:20:18.869976 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.869951    2574 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 19:20:18.870078 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.870025    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pv4mk"
Oct 30 19:20:18.870966 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.870520    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:20:18.873816 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.873798    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tgt5g"
Oct 30 19:20:18.878744 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.878394    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:20:18.878744 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:18.878479    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z7mhx" podUID="725d2cce-2a75-426f-8522-d1ddfbabe1b6"
Oct 30 19:20:18.881189 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.881157    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:18.922859 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.922832    2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 19:20:18.934124 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.934095    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-env-overrides\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:18.934280 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.934130    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ba4ee546-c9c1-4573-b11f-6ba0621972d1-serviceca\") pod \"node-ca-tgt5g\" (UID: \"ba4ee546-c9c1-4573-b11f-6ba0621972d1\") " pod="openshift-image-registry/node-ca-tgt5g"
Oct 30 19:20:18.934280 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.934147    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-cnibin\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:18.934280 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.934165    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-hostroot\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:18.934280 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.934256    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-run-ovn-kubernetes\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:18.934568 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.934287    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-cni-bin\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:18.934568 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.934337    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dls7\" (UniqueName: \"kubernetes.io/projected/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-kube-api-access-5dls7\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:18.934568 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.934399    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1ceff922-50ab-4e0f-b113-03719eb40bc9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:20:18.934568 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.934446    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-sysconfig\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:18.934568 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.934476    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-sysctl-d\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:18.934568 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.934521    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-system-cni-dir\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:18.934867 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.934593    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-var-lib-cni-bin\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:18.934867 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.934644    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-etc-openvswitch\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:18.934867 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.934673    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-run-openvswitch\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:18.934867 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.934701    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1ceff922-50ab-4e0f-b113-03719eb40bc9-cni-binary-copy\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:20:18.934867 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.934732    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bln4w\" (UniqueName: \"kubernetes.io/projected/b41606bf-e8ca-4fc6-8805-b7461187d65e-kube-api-access-bln4w\") pod \"node-resolver-f7tbh\" (UID: \"b41606bf-e8ca-4fc6-8805-b7461187d65e\") " pod="openshift-dns/node-resolver-f7tbh"
Oct 30 19:20:18.934867 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.934759    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2c840c6b-6992-4f29-9023-a0ad99fdc543-agent-certs\") pod \"konnectivity-agent-nw4jx\" (UID: \"2c840c6b-6992-4f29-9023-a0ad99fdc543\") " pod="kube-system/konnectivity-agent-nw4jx"
Oct 30 19:20:18.934867 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.934787    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg2rm\" (UniqueName: \"kubernetes.io/projected/41ff5ee2-70a2-4baf-af4e-00e6575c2e99-kube-api-access-rg2rm\") pod \"network-metrics-daemon-9jk99\" (UID: \"41ff5ee2-70a2-4baf-af4e-00e6575c2e99\") " pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:20:18.934867 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.934820    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rvqr\" (UniqueName: \"kubernetes.io/projected/57769216-dd32-4edc-8e73-b4873caa041d-kube-api-access-4rvqr\") pod \"iptables-alerter-c9vlh\" (UID: \"57769216-dd32-4edc-8e73-b4873caa041d\") " pod="openshift-network-operator/iptables-alerter-c9vlh"
Oct 30 19:20:18.934867 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.934849    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-socket-dir\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:20:18.935340 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.934910    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-kubelet\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:18.935340 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.934937    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/725d2cce-2a75-426f-8522-d1ddfbabe1b6-original-pull-secret\") pod \"global-pull-secret-syncer-z7mhx\" (UID: \"725d2cce-2a75-426f-8522-d1ddfbabe1b6\") " pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:20:18.935340 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.934963    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-modprobe-d\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:18.935340 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.935011    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/57769216-dd32-4edc-8e73-b4873caa041d-iptables-alerter-script\") pod \"iptables-alerter-c9vlh\" (UID: \"57769216-dd32-4edc-8e73-b4873caa041d\") " pod="openshift-network-operator/iptables-alerter-c9vlh"
Oct 30 19:20:18.935340 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.935057    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-sys-fs\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:20:18.935340 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.935124    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-run-ovn\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:18.935340 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.935155    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-systemd-units\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:18.935340 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.935185    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-ovnkube-config\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:18.935340 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.935210    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-run\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:18.935340 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.935235    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-sys\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:18.935340 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.935259    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hcrs\" (UniqueName: \"kubernetes.io/projected/8161e358-9105-4d99-88c1-98c17db09674-kube-api-access-4hcrs\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:20:18.935340 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.935293    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-var-lib-openvswitch\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:18.935340 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.935319    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b41606bf-e8ca-4fc6-8805-b7461187d65e-tmp-dir\") pod \"node-resolver-f7tbh\" (UID: \"b41606bf-e8ca-4fc6-8805-b7461187d65e\") " pod="openshift-dns/node-resolver-f7tbh"
Oct 30 19:20:18.936019 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.935349    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2c840c6b-6992-4f29-9023-a0ad99fdc543-konnectivity-ca\") pod \"konnectivity-agent-nw4jx\" (UID: \"2c840c6b-6992-4f29-9023-a0ad99fdc543\") " pod="kube-system/konnectivity-agent-nw4jx"
Oct 30 19:20:18.936019 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.935380    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/725d2cce-2a75-426f-8522-d1ddfbabe1b6-kubelet-config\") pod \"global-pull-secret-syncer-z7mhx\" (UID: \"725d2cce-2a75-426f-8522-d1ddfbabe1b6\") " pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:20:18.936019 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.935406    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-lib-modules\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:18.936019 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.935448    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-multus-socket-dir-parent\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:18.936019 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.935487    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-run-k8s-cni-cncf-io\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:18.936019 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.935512    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1fda3255-2b22-40a0-870b-7b940977e197-multus-daemon-config\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:18.936019 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.935542    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-ovn-node-metrics-cert\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:18.936019 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.935575    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-ovnkube-script-lib\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:18.936019 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.935604    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-run-netns\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:18.936019 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.935633    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ceff922-50ab-4e0f-b113-03719eb40bc9-system-cni-dir\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:20:18.936019 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.935661    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1ceff922-50ab-4e0f-b113-03719eb40bc9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:20:18.936019 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.935699    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41ff5ee2-70a2-4baf-af4e-00e6575c2e99-metrics-certs\") pod \"network-metrics-daemon-9jk99\" (UID: \"41ff5ee2-70a2-4baf-af4e-00e6575c2e99\") " pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:20:18.936019 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.935725    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-host\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:18.936019 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.935750    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-tuned\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:18.936019 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.935785    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/57769216-dd32-4edc-8e73-b4873caa041d-host-slash\") pod \"iptables-alerter-c9vlh\" (UID: \"57769216-dd32-4edc-8e73-b4873caa041d\") " pod="openshift-network-operator/iptables-alerter-c9vlh"
Oct 30 19:20:18.936019 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.935816    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-run-multus-certs\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:18.936493 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.935842    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-var-lib-kubelet\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:18.936493 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.935872    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-registration-dir\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:20:18.936493 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.935903    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-etc-kubernetes\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:18.936493 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.935931    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/725d2cce-2a75-426f-8522-d1ddfbabe1b6-dbus\") pod \"global-pull-secret-syncer-z7mhx\" (UID: \"725d2cce-2a75-426f-8522-d1ddfbabe1b6\") " pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:20:18.936493 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.935979    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:20:18.936493 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.936010    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jdxg\" (UniqueName: \"kubernetes.io/projected/1fda3255-2b22-40a0-870b-7b940977e197-kube-api-access-5jdxg\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:18.936493 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.936037    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-node-log\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:18.936493 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.936064    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-log-socket\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:18.936493 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.936094    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-cni-netd\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:18.936493 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.936128    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1ceff922-50ab-4e0f-b113-03719eb40bc9-cnibin\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:20:18.936493 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.936186    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba4ee546-c9c1-4573-b11f-6ba0621972d1-host\") pod \"node-ca-tgt5g\" (UID: \"ba4ee546-c9c1-4573-b11f-6ba0621972d1\") " pod="openshift-image-registry/node-ca-tgt5g"
Oct 30 19:20:18.936493 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.936215    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwf55\" (UniqueName: \"kubernetes.io/projected/ba4ee546-c9c1-4573-b11f-6ba0621972d1-kube-api-access-rwf55\") pod \"node-ca-tgt5g\" (UID: \"ba4ee546-c9c1-4573-b11f-6ba0621972d1\") " pod="openshift-image-registry/node-ca-tgt5g"
Oct 30 19:20:18.936493 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.936258    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-systemd\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:18.936493 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.936290    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-multus-cni-dir\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:18.936493 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.936318    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skgdt\" (UniqueName: \"kubernetes.io/projected/1ceff922-50ab-4e0f-b113-03719eb40bc9-kube-api-access-skgdt\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:20:18.936493 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.936346    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-sysctl-conf\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:18.937033 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.936373    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8635205-1f61-4a17-bc1f-7876c5536f9c-tmp\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:18.937033 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.936987    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txjsq\" (UniqueName: \"kubernetes.io/projected/a8635205-1f61-4a17-bc1f-7876c5536f9c-kube-api-access-txjsq\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:18.937143 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.937029    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-device-dir\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:20:18.937143 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.937060    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1ceff922-50ab-4e0f-b113-03719eb40bc9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:20:18.937143 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.937092    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-slash\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:18.937143 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.937119    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-run-netns\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:18.937332 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.937146    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b41606bf-e8ca-4fc6-8805-b7461187d65e-hosts-file\") pod \"node-resolver-f7tbh\" (UID: \"b41606bf-e8ca-4fc6-8805-b7461187d65e\") " pod="openshift-dns/node-resolver-f7tbh"
Oct 30 19:20:18.937332 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.937185    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2xwk\" (UniqueName: \"kubernetes.io/projected/cd444ba1-edf1-49aa-a0cf-b863c93e60be-kube-api-access-c2xwk\") pod \"network-check-target-5j799\" (UID: \"cd444ba1-edf1-49aa-a0cf-b863c93e60be\") " pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:20:18.937332 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.937215    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-kubernetes\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:18.937332 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.937240    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-etc-selinux\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:20:18.937332 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.937271    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-run-systemd\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:18.937332 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.937298    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1fda3255-2b22-40a0-870b-7b940977e197-cni-binary-copy\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:18.937332 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.937325    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-var-lib-cni-multus\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:18.937641 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.937362    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-var-lib-kubelet\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:18.937641 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.937391    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-multus-conf-dir\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:18.937641 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.937420    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:18.953207 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.953155    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-152.ec2.internal" event={"ID":"e58ff4b1131d98fcba47fc4e4f2a3553","Type":"ContainerStarted","Data":"6ac107b4bbb8ab01e09738c400166c19b041992f42a2ea5a87e53022c6317bd2"}
Oct 30 19:20:18.954450 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.954412    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal" event={"ID":"3412174e80de20d346c1e09c794a54c8","Type":"ContainerStarted","Data":"fe16d935efdb60f47b80f91c0cd9440e9da8c1aa652bdf06a543d6ccec9b3708"}
Oct 30 19:20:18.967146 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.967117    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-b7kzg\""
Oct 30 19:20:18.967622 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.967598    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 19:20:18.967846 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.967826    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-gx5x8\""
Oct 30 19:20:18.967954 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.967833    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 19:20:18.968025 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.967976    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 19:20:18.968025 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.968011    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-9smtf\""
Oct 30 19:20:18.971802 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.968702    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 19:20:18.971802 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.968892    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 19:20:18.971802 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.969161    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 19:20:18.971802 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.969192    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bcv2g\""
Oct 30 19:20:18.971802 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.969379    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 19:20:18.971802 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.969600    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-w8vq8\""
Oct 30 19:20:18.971802 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.969733    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 19:20:18.971802 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.969801    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 19:20:18.971802 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.970151    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 19:20:18.971802 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.970369    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hxbtq\""
Oct 30 19:20:18.971802 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.970426    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 19:20:18.971802 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.970449    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gx26w\""
Oct 30 19:20:18.971802 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.970653    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 19:20:18.971802 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.970707    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 19:20:18.971802 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.970730    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 19:20:18.971802 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.970785    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 19:20:18.971802 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.970861    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 19:20:18.971802 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.970940    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 19:20:18.971802 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:18.971006    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 19:20:19.038416 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.038320    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2c840c6b-6992-4f29-9023-a0ad99fdc543-konnectivity-ca\") pod \"konnectivity-agent-nw4jx\" (UID: \"2c840c6b-6992-4f29-9023-a0ad99fdc543\") " pod="kube-system/konnectivity-agent-nw4jx"
Oct 30 19:20:19.038416 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.038368    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/725d2cce-2a75-426f-8522-d1ddfbabe1b6-kubelet-config\") pod \"global-pull-secret-syncer-z7mhx\" (UID: \"725d2cce-2a75-426f-8522-d1ddfbabe1b6\") " pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:20:19.038416 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.038394    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-lib-modules\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:19.038691 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.038545    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-multus-socket-dir-parent\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:19.038691 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.038564    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-lib-modules\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:19.038691 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.038606    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-run-k8s-cni-cncf-io\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:19.038691 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.038634    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1fda3255-2b22-40a0-870b-7b940977e197-multus-daemon-config\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:19.038691 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.038661    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-ovn-node-metrics-cert\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.038691 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.038687    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-ovnkube-script-lib\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.038975 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.038705    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-run-k8s-cni-cncf-io\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:19.038975 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.038715    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-run-netns\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:19.038975 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.038742    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ceff922-50ab-4e0f-b113-03719eb40bc9-system-cni-dir\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:20:19.038975 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.038770    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1ceff922-50ab-4e0f-b113-03719eb40bc9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:20:19.038975 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.038798    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41ff5ee2-70a2-4baf-af4e-00e6575c2e99-metrics-certs\") pod \"network-metrics-daemon-9jk99\" (UID: \"41ff5ee2-70a2-4baf-af4e-00e6575c2e99\") " pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:20:19.038975 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.038807    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-multus-socket-dir-parent\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:19.038975 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.038822    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-host\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:19.038975 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.038848    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-tuned\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:19.038975 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.038856    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-run-netns\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:19.038975 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.038873    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/57769216-dd32-4edc-8e73-b4873caa041d-host-slash\") pod \"iptables-alerter-c9vlh\" (UID: \"57769216-dd32-4edc-8e73-b4873caa041d\") " pod="openshift-network-operator/iptables-alerter-c9vlh"
Oct 30 19:20:19.038975 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.038919    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-run-multus-certs\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:19.038975 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.038948    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-var-lib-kubelet\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:19.038975 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.038975    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-registration-dir\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:20:19.039574 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039013    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-etc-kubernetes\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:19.039574 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039038    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/725d2cce-2a75-426f-8522-d1ddfbabe1b6-dbus\") pod \"global-pull-secret-syncer-z7mhx\" (UID: \"725d2cce-2a75-426f-8522-d1ddfbabe1b6\") " pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:20:19.039574 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039064    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:20:19.039574 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039089    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jdxg\" (UniqueName: \"kubernetes.io/projected/1fda3255-2b22-40a0-870b-7b940977e197-kube-api-access-5jdxg\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:19.039574 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039111    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2c840c6b-6992-4f29-9023-a0ad99fdc543-konnectivity-ca\") pod \"konnectivity-agent-nw4jx\" (UID: \"2c840c6b-6992-4f29-9023-a0ad99fdc543\") " pod="kube-system/konnectivity-agent-nw4jx"
Oct 30 19:20:19.039574 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039114    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-node-log\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.039574 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039165    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-node-log\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.039574 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039183    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-log-socket\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.039574 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039215    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-cni-netd\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.039574 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039221    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ceff922-50ab-4e0f-b113-03719eb40bc9-system-cni-dir\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:20:19.039574 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039244    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1ceff922-50ab-4e0f-b113-03719eb40bc9-cnibin\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:20:19.039574 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039291    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba4ee546-c9c1-4573-b11f-6ba0621972d1-host\") pod \"node-ca-tgt5g\" (UID: \"ba4ee546-c9c1-4573-b11f-6ba0621972d1\") " pod="openshift-image-registry/node-ca-tgt5g"
Oct 30 19:20:19.039574 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039318    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwf55\" (UniqueName: \"kubernetes.io/projected/ba4ee546-c9c1-4573-b11f-6ba0621972d1-kube-api-access-rwf55\") pod \"node-ca-tgt5g\" (UID: \"ba4ee546-c9c1-4573-b11f-6ba0621972d1\") " pod="openshift-image-registry/node-ca-tgt5g"
Oct 30 19:20:19.039574 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039344    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-systemd\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:19.039574 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039369    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-multus-cni-dir\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:19.039574 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039400    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skgdt\" (UniqueName: \"kubernetes.io/projected/1ceff922-50ab-4e0f-b113-03719eb40bc9-kube-api-access-skgdt\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:20:19.039574 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039427    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-sysctl-conf\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:19.040254 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039440    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1fda3255-2b22-40a0-870b-7b940977e197-multus-daemon-config\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:19.040254 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039451    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8635205-1f61-4a17-bc1f-7876c5536f9c-tmp\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:19.040254 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039477    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txjsq\" (UniqueName: \"kubernetes.io/projected/a8635205-1f61-4a17-bc1f-7876c5536f9c-kube-api-access-txjsq\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:19.040254 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039501    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-device-dir\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:20:19.040254 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039525    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1ceff922-50ab-4e0f-b113-03719eb40bc9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:20:19.040254 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039550    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-slash\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.040254 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039576    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-run-netns\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.040254 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039603    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b41606bf-e8ca-4fc6-8805-b7461187d65e-hosts-file\") pod \"node-resolver-f7tbh\" (UID: \"b41606bf-e8ca-4fc6-8805-b7461187d65e\") " pod="openshift-dns/node-resolver-f7tbh"
Oct 30 19:20:19.040254 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039627    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2xwk\" (UniqueName: \"kubernetes.io/projected/cd444ba1-edf1-49aa-a0cf-b863c93e60be-kube-api-access-c2xwk\") pod \"network-check-target-5j799\" (UID: \"cd444ba1-edf1-49aa-a0cf-b863c93e60be\") " pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:20:19.040254 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039656    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-kubernetes\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:19.040254 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039680    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-etc-selinux\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:20:19.040254 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039705    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-run-systemd\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.040254 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039730    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1fda3255-2b22-40a0-870b-7b940977e197-cni-binary-copy\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:19.040254 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039756    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-var-lib-cni-multus\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:19.040254 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039781    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-var-lib-kubelet\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:19.040254 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039781    2574 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 19:20:19.040254 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039810    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-multus-conf-dir\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:19.041208 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039811    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1ceff922-50ab-4e0f-b113-03719eb40bc9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:20:19.041208 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039837    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.041208 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039863    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-env-overrides\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.041208 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039886    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ba4ee546-c9c1-4573-b11f-6ba0621972d1-serviceca\") pod \"node-ca-tgt5g\" (UID: \"ba4ee546-c9c1-4573-b11f-6ba0621972d1\") " pod="openshift-image-registry/node-ca-tgt5g"
Oct 30 19:20:19.041208 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039907    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-cnibin\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:19.041208 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:19.039924    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:20:19.041208 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039992    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-hostroot\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:19.041208 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:19.040034    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41ff5ee2-70a2-4baf-af4e-00e6575c2e99-metrics-certs podName:41ff5ee2-70a2-4baf-af4e-00e6575c2e99 nodeName:}" failed. No retries permitted until 2026-10-30 19:20:19.539995448 +0000 UTC m=+3.096217764 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/41ff5ee2-70a2-4baf-af4e-00e6575c2e99-metrics-certs") pod "network-metrics-daemon-9jk99" (UID: "41ff5ee2-70a2-4baf-af4e-00e6575c2e99") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:20:19.041208 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.040049    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-log-socket\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.041208 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.040092    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-cni-netd\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.041208 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.040094    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-host\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:19.041208 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.040144    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1ceff922-50ab-4e0f-b113-03719eb40bc9-cnibin\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:20:19.041208 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.040151    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba4ee546-c9c1-4573-b11f-6ba0621972d1-host\") pod \"node-ca-tgt5g\" (UID: \"ba4ee546-c9c1-4573-b11f-6ba0621972d1\") " pod="openshift-image-registry/node-ca-tgt5g"
Oct 30 19:20:19.041208 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.040474    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-systemd\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:19.041208 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.040534    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-etc-kubernetes\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:19.041208 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.040581    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-multus-cni-dir\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:19.041208 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.040598    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/57769216-dd32-4edc-8e73-b4873caa041d-host-slash\") pod \"iptables-alerter-c9vlh\" (UID: \"57769216-dd32-4edc-8e73-b4873caa041d\") " pod="openshift-network-operator/iptables-alerter-c9vlh"
Oct 30 19:20:19.041208 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.040656    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-run-multus-certs\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:19.042141 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.040711    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-var-lib-kubelet\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:19.042141 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.040762    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-registration-dir\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:20:19.042141 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.041084    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-sysctl-conf\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:19.042141 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.041467    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/725d2cce-2a75-426f-8522-d1ddfbabe1b6-dbus\") pod \"global-pull-secret-syncer-z7mhx\" (UID: \"725d2cce-2a75-426f-8522-d1ddfbabe1b6\") " pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:20:19.042141 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.041530    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:20:19.042141 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.041603    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-etc-selinux\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:20:19.042141 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.041861    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-device-dir\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:20:19.042477 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.042361    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1ceff922-50ab-4e0f-b113-03719eb40bc9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:20:19.042477 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.042430    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-slash\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.042587 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.042478    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-run-netns\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.042587 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.042537    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b41606bf-e8ca-4fc6-8805-b7461187d65e-hosts-file\") pod \"node-resolver-f7tbh\" (UID: \"b41606bf-e8ca-4fc6-8805-b7461187d65e\") " pod="openshift-dns/node-resolver-f7tbh"
Oct 30 19:20:19.044860 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.039928    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-hostroot\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:19.044860 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.042772    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-run-ovn-kubernetes\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.044860 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.042789    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-kubernetes\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:19.044860 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.042814    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-cni-bin\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.044860 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.042843    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-run-ovn-kubernetes\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.044860 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.042850    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dls7\" (UniqueName: \"kubernetes.io/projected/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-kube-api-access-5dls7\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.044860 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.042881    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1ceff922-50ab-4e0f-b113-03719eb40bc9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:20:19.044860 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.042891    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-multus-conf-dir\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:19.044860 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.042908    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-sysconfig\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:19.044860 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.042928    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-run-systemd\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.044860 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.042951    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-sysctl-d\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:19.044860 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.042979    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-system-cni-dir\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:19.044860 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.043004    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-var-lib-cni-bin\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:19.044860 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.043038    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-etc-openvswitch\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.044860 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.043067    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-run-openvswitch\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.044860 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.043097    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1ceff922-50ab-4e0f-b113-03719eb40bc9-cni-binary-copy\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:20:19.044860 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.043119    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bln4w\" (UniqueName: \"kubernetes.io/projected/b41606bf-e8ca-4fc6-8805-b7461187d65e-kube-api-access-bln4w\") pod \"node-resolver-f7tbh\" (UID: \"b41606bf-e8ca-4fc6-8805-b7461187d65e\") " pod="openshift-dns/node-resolver-f7tbh"
Oct 30 19:20:19.045847 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.043141    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2c840c6b-6992-4f29-9023-a0ad99fdc543-agent-certs\") pod \"konnectivity-agent-nw4jx\" (UID: \"2c840c6b-6992-4f29-9023-a0ad99fdc543\") " pod="kube-system/konnectivity-agent-nw4jx"
Oct 30 19:20:19.045847 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.043170    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg2rm\" (UniqueName: \"kubernetes.io/projected/41ff5ee2-70a2-4baf-af4e-00e6575c2e99-kube-api-access-rg2rm\") pod \"network-metrics-daemon-9jk99\" (UID: \"41ff5ee2-70a2-4baf-af4e-00e6575c2e99\") " pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:20:19.045847 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.043201    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rvqr\" (UniqueName: \"kubernetes.io/projected/57769216-dd32-4edc-8e73-b4873caa041d-kube-api-access-4rvqr\") pod \"iptables-alerter-c9vlh\" (UID: \"57769216-dd32-4edc-8e73-b4873caa041d\") " pod="openshift-network-operator/iptables-alerter-c9vlh"
Oct 30 19:20:19.045847 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.043231    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-socket-dir\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:20:19.045847 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.043263    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-kubelet\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.045847 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.043292    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/725d2cce-2a75-426f-8522-d1ddfbabe1b6-original-pull-secret\") pod \"global-pull-secret-syncer-z7mhx\" (UID: \"725d2cce-2a75-426f-8522-d1ddfbabe1b6\") " pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:20:19.045847 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.043323    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-modprobe-d\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:19.045847 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.043351    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/57769216-dd32-4edc-8e73-b4873caa041d-iptables-alerter-script\") pod \"iptables-alerter-c9vlh\" (UID: \"57769216-dd32-4edc-8e73-b4873caa041d\") " pod="openshift-network-operator/iptables-alerter-c9vlh"
Oct 30 19:20:19.045847 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.043381    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-sys-fs\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:20:19.045847 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.043408    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-run-ovn\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.045847 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.043435    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-systemd-units\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.045847 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.043447    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-ovnkube-script-lib\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.045847 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.043462    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-ovnkube-config\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.045847 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.043503    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-run\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:19.045847 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.043530    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-sys\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:19.045847 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.043559    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hcrs\" (UniqueName: \"kubernetes.io/projected/8161e358-9105-4d99-88c1-98c17db09674-kube-api-access-4hcrs\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:20:19.045847 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.043586    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-var-lib-openvswitch\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.046786 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.043615    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b41606bf-e8ca-4fc6-8805-b7461187d65e-tmp-dir\") pod \"node-resolver-f7tbh\" (UID: \"b41606bf-e8ca-4fc6-8805-b7461187d65e\") " pod="openshift-dns/node-resolver-f7tbh"
Oct 30 19:20:19.046786 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.043950    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b41606bf-e8ca-4fc6-8805-b7461187d65e-tmp-dir\") pod \"node-resolver-f7tbh\" (UID: \"b41606bf-e8ca-4fc6-8805-b7461187d65e\") " pod="openshift-dns/node-resolver-f7tbh"
Oct 30 19:20:19.046786 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.044032    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-run\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:19.046786 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.044082    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-sys\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:19.046786 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.044247    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-var-lib-openvswitch\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.046786 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.044311    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-var-lib-cni-multus\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:19.046786 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:19.044404    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 19:20:19.046786 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:19.044477    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/725d2cce-2a75-426f-8522-d1ddfbabe1b6-original-pull-secret podName:725d2cce-2a75-426f-8522-d1ddfbabe1b6 nodeName:}" failed. No retries permitted until 2026-10-30 19:20:19.544457238 +0000 UTC m=+3.100679511 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/725d2cce-2a75-426f-8522-d1ddfbabe1b6-original-pull-secret") pod "global-pull-secret-syncer-z7mhx" (UID: "725d2cce-2a75-426f-8522-d1ddfbabe1b6") : object "kube-system"/"original-pull-secret" not registered
Oct 30 19:20:19.046786 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.044493    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-cni-bin\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.046786 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.044605    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-ovnkube-config\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.046786 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.044709    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-modprobe-d\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:19.046786 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.044738    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-ovn-node-metrics-cert\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.046786 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.044837    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1ceff922-50ab-4e0f-b113-03719eb40bc9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:20:19.046786 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.044843    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-var-lib-kubelet\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:19.046786 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.044914    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-sysconfig\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:19.046786 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.045033    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-sysctl-d\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:19.046786 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.045095    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-system-cni-dir\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:19.046786 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.045144    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-var-lib-cni-bin\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:19.047725 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.045187    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-etc-openvswitch\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.047725 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.045228    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-env-overrides\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.047725 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.045232    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-run-openvswitch\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.047725 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.045302    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.047725 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.045306    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/57769216-dd32-4edc-8e73-b4873caa041d-iptables-alerter-script\") pod \"iptables-alerter-c9vlh\" (UID: \"57769216-dd32-4edc-8e73-b4873caa041d\") " pod="openshift-network-operator/iptables-alerter-c9vlh"
Oct 30 19:20:19.047725 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.045355    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-sys-fs\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:20:19.047725 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.045401    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-run-ovn\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.047725 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.045440    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-systemd-units\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.047725 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.045471    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1fda3255-2b22-40a0-870b-7b940977e197-cni-binary-copy\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:19.047725 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.045709    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1ceff922-50ab-4e0f-b113-03719eb40bc9-cni-binary-copy\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:20:19.047725 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.045818    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-socket-dir\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:20:19.047725 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.045862    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-cnibin\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:19.047725 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.045877    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-kubelet\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.047725 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.046114    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8635205-1f61-4a17-bc1f-7876c5536f9c-tmp\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:19.047725 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.046279    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ba4ee546-c9c1-4573-b11f-6ba0621972d1-serviceca\") pod \"node-ca-tgt5g\" (UID: \"ba4ee546-c9c1-4573-b11f-6ba0621972d1\") " pod="openshift-image-registry/node-ca-tgt5g"
Oct 30 19:20:19.047725 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.046401    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/725d2cce-2a75-426f-8522-d1ddfbabe1b6-kubelet-config\") pod \"global-pull-secret-syncer-z7mhx\" (UID: \"725d2cce-2a75-426f-8522-d1ddfbabe1b6\") " pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:20:19.049527 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.049499    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-tuned\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:19.051434 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.051384    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2c840c6b-6992-4f29-9023-a0ad99fdc543-agent-certs\") pod \"konnectivity-agent-nw4jx\" (UID: \"2c840c6b-6992-4f29-9023-a0ad99fdc543\") " pod="kube-system/konnectivity-agent-nw4jx"
Oct 30 19:20:19.059115 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:19.059091    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 19:20:19.059115 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:19.059116    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 19:20:19.059303 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:19.059131    2574 projected.go:196] Error preparing data for projected volume kube-api-access-c2xwk for pod openshift-network-diagnostics/network-check-target-5j799: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:20:19.059303 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:19.059204    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd444ba1-edf1-49aa-a0cf-b863c93e60be-kube-api-access-c2xwk podName:cd444ba1-edf1-49aa-a0cf-b863c93e60be nodeName:}" failed. No retries permitted until 2026-10-30 19:20:19.559184286 +0000 UTC m=+3.115406562 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-c2xwk" (UniqueName: "kubernetes.io/projected/cd444ba1-edf1-49aa-a0cf-b863c93e60be-kube-api-access-c2xwk") pod "network-check-target-5j799" (UID: "cd444ba1-edf1-49aa-a0cf-b863c93e60be") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:20:19.060666 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.060322    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwf55\" (UniqueName: \"kubernetes.io/projected/ba4ee546-c9c1-4573-b11f-6ba0621972d1-kube-api-access-rwf55\") pod \"node-ca-tgt5g\" (UID: \"ba4ee546-c9c1-4573-b11f-6ba0621972d1\") " pod="openshift-image-registry/node-ca-tgt5g"
Oct 30 19:20:19.061206 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.061186    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-skgdt\" (UniqueName: \"kubernetes.io/projected/1ceff922-50ab-4e0f-b113-03719eb40bc9-kube-api-access-skgdt\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:20:19.062675 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.062145    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txjsq\" (UniqueName: \"kubernetes.io/projected/a8635205-1f61-4a17-bc1f-7876c5536f9c-kube-api-access-txjsq\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:19.062898 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.062874    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jdxg\" (UniqueName: \"kubernetes.io/projected/1fda3255-2b22-40a0-870b-7b940977e197-kube-api-access-5jdxg\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:19.064190 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.064064    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hcrs\" (UniqueName: \"kubernetes.io/projected/8161e358-9105-4d99-88c1-98c17db09674-kube-api-access-4hcrs\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:20:19.065324 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.065286    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bln4w\" (UniqueName: \"kubernetes.io/projected/b41606bf-e8ca-4fc6-8805-b7461187d65e-kube-api-access-bln4w\") pod \"node-resolver-f7tbh\" (UID: \"b41606bf-e8ca-4fc6-8805-b7461187d65e\") " pod="openshift-dns/node-resolver-f7tbh"
Oct 30 19:20:19.066630 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.066561    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg2rm\" (UniqueName: \"kubernetes.io/projected/41ff5ee2-70a2-4baf-af4e-00e6575c2e99-kube-api-access-rg2rm\") pod \"network-metrics-daemon-9jk99\" (UID: \"41ff5ee2-70a2-4baf-af4e-00e6575c2e99\") " pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:20:19.066630 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.066583    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dls7\" (UniqueName: \"kubernetes.io/projected/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-kube-api-access-5dls7\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.067193 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.067172    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rvqr\" (UniqueName: \"kubernetes.io/projected/57769216-dd32-4edc-8e73-b4873caa041d-kube-api-access-4rvqr\") pod \"iptables-alerter-c9vlh\" (UID: \"57769216-dd32-4edc-8e73-b4873caa041d\") " pod="openshift-network-operator/iptables-alerter-c9vlh"
Oct 30 19:20:19.179784 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.179514    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-f7tbh"
Oct 30 19:20:19.185660 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.185636    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:19.193854 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:19.193827    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb41606bf_e8ca_4fc6_8805_b7461187d65e.slice/crio-399dc2728fd6bfe75e8e8f5ea5a208feb42abca5755c9ec2323bdc27e2afc8a5 WatchSource:0}: Error finding container 399dc2728fd6bfe75e8e8f5ea5a208feb42abca5755c9ec2323bdc27e2afc8a5: Status 404 returned error can't find the container with id 399dc2728fd6bfe75e8e8f5ea5a208feb42abca5755c9ec2323bdc27e2afc8a5
Oct 30 19:20:19.195186 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.195163    2574 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:20:19.198284 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:19.198242    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23ec668e_b2fb_471f_a7c7_13095a3dcf7f.slice/crio-d007f9d11dde4a56571ed3b2441f4f3d884c8bfeb549eecd1a362a77805a1420 WatchSource:0}: Error finding container d007f9d11dde4a56571ed3b2441f4f3d884c8bfeb549eecd1a362a77805a1420: Status 404 returned error can't find the container with id d007f9d11dde4a56571ed3b2441f4f3d884c8bfeb549eecd1a362a77805a1420
Oct 30 19:20:19.201513 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.201491    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nw4jx"
Oct 30 19:20:19.202907 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.202883    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:20:19.207819 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.207773    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:20:19.210461 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.210439    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-c9vlh"
Oct 30 19:20:19.216515 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:19.216480    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c840c6b_6992_4f29_9023_a0ad99fdc543.slice/crio-c97b572cfac5edb55782195f4e1400e2bbec702087499bfdac85303de5ba47a5 WatchSource:0}: Error finding container c97b572cfac5edb55782195f4e1400e2bbec702087499bfdac85303de5ba47a5: Status 404 returned error can't find the container with id c97b572cfac5edb55782195f4e1400e2bbec702087499bfdac85303de5ba47a5
Oct 30 19:20:19.217483 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:19.217457    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8635205_1f61_4a17_bc1f_7876c5536f9c.slice/crio-8ec9cd61f5472ec090d470c36989b12e6c9e68f4a57d8acecd4b34f26320093d WatchSource:0}: Error finding container 8ec9cd61f5472ec090d470c36989b12e6c9e68f4a57d8acecd4b34f26320093d: Status 404 returned error can't find the container with id 8ec9cd61f5472ec090d470c36989b12e6c9e68f4a57d8acecd4b34f26320093d
Oct 30 19:20:19.222539 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.222415    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:20:19.225468 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:19.225443    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57769216_dd32_4edc_8e73_b4873caa041d.slice/crio-a7ab6ba877af77a68f6d36be8910e84f896e34840a6913fa1266f5a943f91917 WatchSource:0}: Error finding container a7ab6ba877af77a68f6d36be8910e84f896e34840a6913fa1266f5a943f91917: Status 404 returned error can't find the container with id a7ab6ba877af77a68f6d36be8910e84f896e34840a6913fa1266f5a943f91917
Oct 30 19:20:19.226355 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.226284    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tgt5g"
Oct 30 19:20:19.230275 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.230252    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-sdlcz"
Oct 30 19:20:19.235374 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:19.235342    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8161e358_9105_4d99_88c1_98c17db09674.slice/crio-88bd605a5c6886f91bc53cf97ae5477a38e46ac2fb529ef77ac3fe8cb5b6cdfd WatchSource:0}: Error finding container 88bd605a5c6886f91bc53cf97ae5477a38e46ac2fb529ef77ac3fe8cb5b6cdfd: Status 404 returned error can't find the container with id 88bd605a5c6886f91bc53cf97ae5477a38e46ac2fb529ef77ac3fe8cb5b6cdfd
Oct 30 19:20:19.242654 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:19.242625    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba4ee546_c9c1_4573_b11f_6ba0621972d1.slice/crio-79af4843b7a47f58635d1ee037528e4872d865f5a232bb6cbe3589c670b5197d WatchSource:0}: Error finding container 79af4843b7a47f58635d1ee037528e4872d865f5a232bb6cbe3589c670b5197d: Status 404 returned error can't find the container with id 79af4843b7a47f58635d1ee037528e4872d865f5a232bb6cbe3589c670b5197d
Oct 30 19:20:19.249849 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:20:19.249821    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fda3255_2b22_40a0_870b_7b940977e197.slice/crio-1ecb2762907f7b9889572ec183e48157056de637daf0bd21003cc49cb96208ef WatchSource:0}: Error finding container 1ecb2762907f7b9889572ec183e48157056de637daf0bd21003cc49cb96208ef: Status 404 returned error can't find the container with id 1ecb2762907f7b9889572ec183e48157056de637daf0bd21003cc49cb96208ef
Oct 30 19:20:19.547467 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.547428    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/725d2cce-2a75-426f-8522-d1ddfbabe1b6-original-pull-secret\") pod \"global-pull-secret-syncer-z7mhx\" (UID: \"725d2cce-2a75-426f-8522-d1ddfbabe1b6\") " pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:20:19.547467 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.547477    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41ff5ee2-70a2-4baf-af4e-00e6575c2e99-metrics-certs\") pod \"network-metrics-daemon-9jk99\" (UID: \"41ff5ee2-70a2-4baf-af4e-00e6575c2e99\") " pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:20:19.547713 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:19.547599    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:20:19.547713 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:19.547619    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 19:20:19.547713 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:19.547662    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41ff5ee2-70a2-4baf-af4e-00e6575c2e99-metrics-certs podName:41ff5ee2-70a2-4baf-af4e-00e6575c2e99 nodeName:}" failed. No retries permitted until 2026-10-30 19:20:20.547644133 +0000 UTC m=+4.103866392 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/41ff5ee2-70a2-4baf-af4e-00e6575c2e99-metrics-certs") pod "network-metrics-daemon-9jk99" (UID: "41ff5ee2-70a2-4baf-af4e-00e6575c2e99") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:20:19.547713 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:19.547689    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/725d2cce-2a75-426f-8522-d1ddfbabe1b6-original-pull-secret podName:725d2cce-2a75-426f-8522-d1ddfbabe1b6 nodeName:}" failed. No retries permitted until 2026-10-30 19:20:20.547673229 +0000 UTC m=+4.103895499 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/725d2cce-2a75-426f-8522-d1ddfbabe1b6-original-pull-secret") pod "global-pull-secret-syncer-z7mhx" (UID: "725d2cce-2a75-426f-8522-d1ddfbabe1b6") : object "kube-system"/"original-pull-secret" not registered
Oct 30 19:20:19.648116 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.648088    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2xwk\" (UniqueName: \"kubernetes.io/projected/cd444ba1-edf1-49aa-a0cf-b863c93e60be-kube-api-access-c2xwk\") pod \"network-check-target-5j799\" (UID: \"cd444ba1-edf1-49aa-a0cf-b863c93e60be\") " pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:20:19.648282 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:19.648228    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 19:20:19.648282 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:19.648249    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 19:20:19.648282 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:19.648271    2574 projected.go:196] Error preparing data for projected volume kube-api-access-c2xwk for pod openshift-network-diagnostics/network-check-target-5j799: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:20:19.648447 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:19.648324    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd444ba1-edf1-49aa-a0cf-b863c93e60be-kube-api-access-c2xwk podName:cd444ba1-edf1-49aa-a0cf-b863c93e60be nodeName:}" failed. No retries permitted until 2026-10-30 19:20:20.648310923 +0000 UTC m=+4.204533180 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-c2xwk" (UniqueName: "kubernetes.io/projected/cd444ba1-edf1-49aa-a0cf-b863c93e60be-kube-api-access-c2xwk") pod "network-check-target-5j799" (UID: "cd444ba1-edf1-49aa-a0cf-b863c93e60be") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:20:19.873927 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.873603    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 19:15:19 +0000 UTC" deadline="2028-04-22 04:27:15.559526391 +0000 UTC"
Oct 30 19:20:19.873927 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.873927    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12945h6m55.685606313s"
Oct 30 19:20:19.964832 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.964705    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nw4jx" event={"ID":"2c840c6b-6992-4f29-9023-a0ad99fdc543","Type":"ContainerStarted","Data":"c97b572cfac5edb55782195f4e1400e2bbec702087499bfdac85303de5ba47a5"}
Oct 30 19:20:19.966832 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.966780    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8pv94" event={"ID":"23ec668e-b2fb-471f-a7c7-13095a3dcf7f","Type":"ContainerStarted","Data":"d007f9d11dde4a56571ed3b2441f4f3d884c8bfeb549eecd1a362a77805a1420"}
Oct 30 19:20:19.968542 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.968513    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-f7tbh" event={"ID":"b41606bf-e8ca-4fc6-8805-b7461187d65e","Type":"ContainerStarted","Data":"399dc2728fd6bfe75e8e8f5ea5a208feb42abca5755c9ec2323bdc27e2afc8a5"}
Oct 30 19:20:19.975020 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.974992    2574 generic.go:356] "Generic (PLEG): container finished" podID="3412174e80de20d346c1e09c794a54c8" containerID="4babb5839e8b3ac96725738008c894105d2676a536e4897b2e08341aa8ca7cd3" exitCode=0
Oct 30 19:20:19.975153 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.975101    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal" event={"ID":"3412174e80de20d346c1e09c794a54c8","Type":"ContainerDied","Data":"4babb5839e8b3ac96725738008c894105d2676a536e4897b2e08341aa8ca7cd3"}
Oct 30 19:20:19.985142 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.985110    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tgt5g" event={"ID":"ba4ee546-c9c1-4573-b11f-6ba0621972d1","Type":"ContainerStarted","Data":"79af4843b7a47f58635d1ee037528e4872d865f5a232bb6cbe3589c670b5197d"}
Oct 30 19:20:19.988592 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:19.988542    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch" event={"ID":"8161e358-9105-4d99-88c1-98c17db09674","Type":"ContainerStarted","Data":"88bd605a5c6886f91bc53cf97ae5477a38e46ac2fb529ef77ac3fe8cb5b6cdfd"}
Oct 30 19:20:20.006769 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:20.006228    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-152.ec2.internal" event={"ID":"e58ff4b1131d98fcba47fc4e4f2a3553","Type":"ContainerStarted","Data":"1b6d05a757c1cc5257fcad2ae658ea4291fcfba280e72e9942db24513330ae12"}
Oct 30 19:20:20.015269 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:20.015141    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-sdlcz" event={"ID":"1fda3255-2b22-40a0-870b-7b940977e197","Type":"ContainerStarted","Data":"1ecb2762907f7b9889572ec183e48157056de637daf0bd21003cc49cb96208ef"}
Oct 30 19:20:20.020031 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:20.019951    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-c9vlh" event={"ID":"57769216-dd32-4edc-8e73-b4873caa041d","Type":"ContainerStarted","Data":"a7ab6ba877af77a68f6d36be8910e84f896e34840a6913fa1266f5a943f91917"}
Oct 30 19:20:20.022998 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:20.022967    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hcmmq" event={"ID":"1ceff922-50ab-4e0f-b113-03719eb40bc9","Type":"ContainerStarted","Data":"8e6694fb5f64111134a16b0c70a9d467732ebe834c694bb55f0c657b98a0c1fb"}
Oct 30 19:20:20.025462 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:20.025433    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dvzwm" event={"ID":"a8635205-1f61-4a17-bc1f-7876c5536f9c","Type":"ContainerStarted","Data":"8ec9cd61f5472ec090d470c36989b12e6c9e68f4a57d8acecd4b34f26320093d"}
Oct 30 19:20:20.555220 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:20.555182    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/725d2cce-2a75-426f-8522-d1ddfbabe1b6-original-pull-secret\") pod \"global-pull-secret-syncer-z7mhx\" (UID: \"725d2cce-2a75-426f-8522-d1ddfbabe1b6\") " pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:20:20.555425 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:20.555247    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41ff5ee2-70a2-4baf-af4e-00e6575c2e99-metrics-certs\") pod \"network-metrics-daemon-9jk99\" (UID: \"41ff5ee2-70a2-4baf-af4e-00e6575c2e99\") " pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:20:20.555425 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:20.555402    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:20:20.555541 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:20.555464    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41ff5ee2-70a2-4baf-af4e-00e6575c2e99-metrics-certs podName:41ff5ee2-70a2-4baf-af4e-00e6575c2e99 nodeName:}" failed. No retries permitted until 2026-10-30 19:20:22.555446019 +0000 UTC m=+6.111668279 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/41ff5ee2-70a2-4baf-af4e-00e6575c2e99-metrics-certs") pod "network-metrics-daemon-9jk99" (UID: "41ff5ee2-70a2-4baf-af4e-00e6575c2e99") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:20:20.555783 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:20.555766    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 19:20:20.555854 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:20.555809    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/725d2cce-2a75-426f-8522-d1ddfbabe1b6-original-pull-secret podName:725d2cce-2a75-426f-8522-d1ddfbabe1b6 nodeName:}" failed. No retries permitted until 2026-10-30 19:20:22.555798562 +0000 UTC m=+6.112020822 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/725d2cce-2a75-426f-8522-d1ddfbabe1b6-original-pull-secret") pod "global-pull-secret-syncer-z7mhx" (UID: "725d2cce-2a75-426f-8522-d1ddfbabe1b6") : object "kube-system"/"original-pull-secret" not registered
Oct 30 19:20:20.656500 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:20.655592    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2xwk\" (UniqueName: \"kubernetes.io/projected/cd444ba1-edf1-49aa-a0cf-b863c93e60be-kube-api-access-c2xwk\") pod \"network-check-target-5j799\" (UID: \"cd444ba1-edf1-49aa-a0cf-b863c93e60be\") " pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:20:20.656500 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:20.655779    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 19:20:20.656500 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:20.655798    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 19:20:20.656500 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:20.655812    2574 projected.go:196] Error preparing data for projected volume kube-api-access-c2xwk for pod openshift-network-diagnostics/network-check-target-5j799: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:20:20.656500 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:20.655870    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd444ba1-edf1-49aa-a0cf-b863c93e60be-kube-api-access-c2xwk podName:cd444ba1-edf1-49aa-a0cf-b863c93e60be nodeName:}" failed. No retries permitted until 2026-10-30 19:20:22.655852061 +0000 UTC m=+6.212074333 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-c2xwk" (UniqueName: "kubernetes.io/projected/cd444ba1-edf1-49aa-a0cf-b863c93e60be-kube-api-access-c2xwk") pod "network-check-target-5j799" (UID: "cd444ba1-edf1-49aa-a0cf-b863c93e60be") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:20:20.874669 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:20.874564    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 19:15:19 +0000 UTC" deadline="2028-05-29 13:48:28.778447744 +0000 UTC"
Oct 30 19:20:20.874669 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:20.874617    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13842h28m7.90383558s"
Oct 30 19:20:20.955544 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:20.955476    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:20:20.956024 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:20.955988    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z7mhx" podUID="725d2cce-2a75-426f-8522-d1ddfbabe1b6"
Oct 30 19:20:20.970176 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:20.969037    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:20:20.970176 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:20.969239    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5j799" podUID="cd444ba1-edf1-49aa-a0cf-b863c93e60be"
Oct 30 19:20:20.970176 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:20.969346    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:20:20.970176 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:20.969494    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jk99" podUID="41ff5ee2-70a2-4baf-af4e-00e6575c2e99"
Oct 30 19:20:21.036092 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:21.035973    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal" event={"ID":"3412174e80de20d346c1e09c794a54c8","Type":"ContainerStarted","Data":"58206b1f6bf6af344b08f85b7cd912b8f0268072a20ccb6fc5161b9570bc827c"}
Oct 30 19:20:21.076096 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:21.074355    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-152.ec2.internal" podStartSLOduration=3.074337173 podStartE2EDuration="3.074337173s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:20:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:20:20.053278399 +0000 UTC m=+3.609500681" watchObservedRunningTime="2026-10-30 19:20:21.074337173 +0000 UTC m=+4.630559454"
Oct 30 19:20:22.583947 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:22.583838    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/725d2cce-2a75-426f-8522-d1ddfbabe1b6-original-pull-secret\") pod \"global-pull-secret-syncer-z7mhx\" (UID: \"725d2cce-2a75-426f-8522-d1ddfbabe1b6\") " pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:20:22.583947 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:22.583928    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41ff5ee2-70a2-4baf-af4e-00e6575c2e99-metrics-certs\") pod \"network-metrics-daemon-9jk99\" (UID: \"41ff5ee2-70a2-4baf-af4e-00e6575c2e99\") " pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:20:22.584522 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:22.584070    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:20:22.584522 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:22.584200    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41ff5ee2-70a2-4baf-af4e-00e6575c2e99-metrics-certs podName:41ff5ee2-70a2-4baf-af4e-00e6575c2e99 nodeName:}" failed. No retries permitted until 2026-10-30 19:20:26.584119838 +0000 UTC m=+10.140342114 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/41ff5ee2-70a2-4baf-af4e-00e6575c2e99-metrics-certs") pod "network-metrics-daemon-9jk99" (UID: "41ff5ee2-70a2-4baf-af4e-00e6575c2e99") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:20:22.584690 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:22.584669    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 19:20:22.584752 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:22.584731    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/725d2cce-2a75-426f-8522-d1ddfbabe1b6-original-pull-secret podName:725d2cce-2a75-426f-8522-d1ddfbabe1b6 nodeName:}" failed. No retries permitted until 2026-10-30 19:20:26.584715659 +0000 UTC m=+10.140938134 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/725d2cce-2a75-426f-8522-d1ddfbabe1b6-original-pull-secret") pod "global-pull-secret-syncer-z7mhx" (UID: "725d2cce-2a75-426f-8522-d1ddfbabe1b6") : object "kube-system"/"original-pull-secret" not registered
Oct 30 19:20:22.685734 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:22.684967    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2xwk\" (UniqueName: \"kubernetes.io/projected/cd444ba1-edf1-49aa-a0cf-b863c93e60be-kube-api-access-c2xwk\") pod \"network-check-target-5j799\" (UID: \"cd444ba1-edf1-49aa-a0cf-b863c93e60be\") " pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:20:22.685734 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:22.685197    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 19:20:22.685734 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:22.685220    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 19:20:22.685734 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:22.685243    2574 projected.go:196] Error preparing data for projected volume kube-api-access-c2xwk for pod openshift-network-diagnostics/network-check-target-5j799: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:20:22.685734 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:22.685342    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd444ba1-edf1-49aa-a0cf-b863c93e60be-kube-api-access-c2xwk podName:cd444ba1-edf1-49aa-a0cf-b863c93e60be nodeName:}" failed. No retries permitted until 2026-10-30 19:20:26.685321358 +0000 UTC m=+10.241543632 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-c2xwk" (UniqueName: "kubernetes.io/projected/cd444ba1-edf1-49aa-a0cf-b863c93e60be-kube-api-access-c2xwk") pod "network-check-target-5j799" (UID: "cd444ba1-edf1-49aa-a0cf-b863c93e60be") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:20:22.949499 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:22.949420    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:20:22.949664 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:22.949564    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5j799" podUID="cd444ba1-edf1-49aa-a0cf-b863c93e60be"
Oct 30 19:20:22.950016 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:22.949994    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:20:22.950198 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:22.950173    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jk99" podUID="41ff5ee2-70a2-4baf-af4e-00e6575c2e99"
Oct 30 19:20:22.950296 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:22.950246    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:20:22.950365 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:22.950344    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z7mhx" podUID="725d2cce-2a75-426f-8522-d1ddfbabe1b6"
Oct 30 19:20:23.047824 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:23.047770    2574 generic.go:356] "Generic (PLEG): container finished" podID="1ceff922-50ab-4e0f-b113-03719eb40bc9" containerID="2a29f47271063b29b11594ec378707aeb8b1089c9ea2b9a32b53495dabe7468c" exitCode=0
Oct 30 19:20:23.048042 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:23.047839    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hcmmq" event={"ID":"1ceff922-50ab-4e0f-b113-03719eb40bc9","Type":"ContainerDied","Data":"2a29f47271063b29b11594ec378707aeb8b1089c9ea2b9a32b53495dabe7468c"}
Oct 30 19:20:23.117117 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:23.115615    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal" podStartSLOduration=5.115595136 podStartE2EDuration="5.115595136s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:20:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:20:21.075699084 +0000 UTC m=+4.631921367" watchObservedRunningTime="2026-10-30 19:20:23.115595136 +0000 UTC m=+6.671817419"
Oct 30 19:20:24.058582 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:24.058536    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nw4jx" event={"ID":"2c840c6b-6992-4f29-9023-a0ad99fdc543","Type":"ContainerStarted","Data":"dcbaa0974bd969467dfb300c765d52952b808bde9a63f383f225dfa559904481"}
Oct 30 19:20:24.094797 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:24.093741    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-nw4jx" podStartSLOduration=2.799164725 podStartE2EDuration="7.093720248s" totalImagesPullingTime="4.294555523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:20:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:20:24.093353021 +0000 UTC m=+7.649575291" watchObservedRunningTime="2026-10-30 19:20:24.093720248 +0000 UTC m=+7.649942526"
Oct 30 19:20:24.949337 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:24.949305    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:20:24.949503 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:24.949435    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5j799" podUID="cd444ba1-edf1-49aa-a0cf-b863c93e60be"
Oct 30 19:20:24.949853 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:24.949835    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:20:24.949987 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:24.949960    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jk99" podUID="41ff5ee2-70a2-4baf-af4e-00e6575c2e99"
Oct 30 19:20:24.950059 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:24.950039    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:20:24.950135 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:24.950117    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z7mhx" podUID="725d2cce-2a75-426f-8522-d1ddfbabe1b6"
Oct 30 19:20:25.074672 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:25.074642    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tgt5g" event={"ID":"ba4ee546-c9c1-4573-b11f-6ba0621972d1","Type":"ContainerStarted","Data":"6e52b1a43916176730189c4aa33b289430c25d9bd800e712c0ec6b167a24b8d5"}
Oct 30 19:20:25.109104 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:25.108901    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-tgt5g" podStartSLOduration=2.936450647 podStartE2EDuration="8.108886252s" totalImagesPullingTime="5.172435605s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:20:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:20:25.108704591 +0000 UTC m=+8.664926891" watchObservedRunningTime="2026-10-30 19:20:25.108886252 +0000 UTC m=+8.665108528"
Oct 30 19:20:26.079640 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:26.079606    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch" event={"ID":"8161e358-9105-4d99-88c1-98c17db09674","Type":"ContainerStarted","Data":"e40e324c9bcbcb43d35f2263f493cd7ee796afd6b5a1f9d1cda20c4c9f1121cc"}
Oct 30 19:20:26.622312 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:26.622274    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/725d2cce-2a75-426f-8522-d1ddfbabe1b6-original-pull-secret\") pod \"global-pull-secret-syncer-z7mhx\" (UID: \"725d2cce-2a75-426f-8522-d1ddfbabe1b6\") " pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:20:26.622494 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:26.622338    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41ff5ee2-70a2-4baf-af4e-00e6575c2e99-metrics-certs\") pod \"network-metrics-daemon-9jk99\" (UID: \"41ff5ee2-70a2-4baf-af4e-00e6575c2e99\") " pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:20:26.622494 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:26.622470    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:20:26.622494 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:26.622471    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 19:20:26.622656 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:26.622534    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41ff5ee2-70a2-4baf-af4e-00e6575c2e99-metrics-certs podName:41ff5ee2-70a2-4baf-af4e-00e6575c2e99 nodeName:}" failed. No retries permitted until 2026-10-30 19:20:34.622516577 +0000 UTC m=+18.178738834 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/41ff5ee2-70a2-4baf-af4e-00e6575c2e99-metrics-certs") pod "network-metrics-daemon-9jk99" (UID: "41ff5ee2-70a2-4baf-af4e-00e6575c2e99") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:20:26.622656 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:26.622557    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/725d2cce-2a75-426f-8522-d1ddfbabe1b6-original-pull-secret podName:725d2cce-2a75-426f-8522-d1ddfbabe1b6 nodeName:}" failed. No retries permitted until 2026-10-30 19:20:34.622546585 +0000 UTC m=+18.178768843 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/725d2cce-2a75-426f-8522-d1ddfbabe1b6-original-pull-secret") pod "global-pull-secret-syncer-z7mhx" (UID: "725d2cce-2a75-426f-8522-d1ddfbabe1b6") : object "kube-system"/"original-pull-secret" not registered
Oct 30 19:20:26.723684 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:26.723651    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2xwk\" (UniqueName: \"kubernetes.io/projected/cd444ba1-edf1-49aa-a0cf-b863c93e60be-kube-api-access-c2xwk\") pod \"network-check-target-5j799\" (UID: \"cd444ba1-edf1-49aa-a0cf-b863c93e60be\") " pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:20:26.724326 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:26.723848    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 19:20:26.724326 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:26.723890    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 19:20:26.724326 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:26.723903    2574 projected.go:196] Error preparing data for projected volume kube-api-access-c2xwk for pod openshift-network-diagnostics/network-check-target-5j799: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:20:26.724326 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:26.723967    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd444ba1-edf1-49aa-a0cf-b863c93e60be-kube-api-access-c2xwk podName:cd444ba1-edf1-49aa-a0cf-b863c93e60be nodeName:}" failed. No retries permitted until 2026-10-30 19:20:34.723948418 +0000 UTC m=+18.280170696 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-c2xwk" (UniqueName: "kubernetes.io/projected/cd444ba1-edf1-49aa-a0cf-b863c93e60be-kube-api-access-c2xwk") pod "network-check-target-5j799" (UID: "cd444ba1-edf1-49aa-a0cf-b863c93e60be") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:20:26.863120 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:26.863017    2574 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 19:20:26.899162 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:26.898616    2574 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T19:20:26.863038778Z","UUID":"7672fa75-dd3a-4333-8dc7-d7609764d6bb","Handler":null,"Name":"","Endpoint":""}
Oct 30 19:20:26.900590 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:26.900026    2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 19:20:26.900590 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:26.900055    2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 19:20:26.949837 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:26.949432    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:20:26.949837 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:26.949558    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5j799" podUID="cd444ba1-edf1-49aa-a0cf-b863c93e60be"
Oct 30 19:20:26.949837 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:26.949580    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:20:26.949837 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:26.949715    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jk99" podUID="41ff5ee2-70a2-4baf-af4e-00e6575c2e99"
Oct 30 19:20:26.949837 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:26.949728    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:20:26.950123 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:26.949992    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z7mhx" podUID="725d2cce-2a75-426f-8522-d1ddfbabe1b6"
Oct 30 19:20:27.085197 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:27.085093    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch" event={"ID":"8161e358-9105-4d99-88c1-98c17db09674","Type":"ContainerStarted","Data":"84288a0c3111905d2eb7f74ae6402306b2f35bea97cbb5ee51ed21b2a6e83b57"}
Oct 30 19:20:27.088554 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:27.087892    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-sdlcz" event={"ID":"1fda3255-2b22-40a0-870b-7b940977e197","Type":"ContainerStarted","Data":"de7f90a832e3cda9a8e275f06705e66d9dd422cff93833cafad0cb38109ef73c"}
Oct 30 19:20:27.129032 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:27.128977    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-sdlcz" podStartSLOduration=2.550234948 podStartE2EDuration="10.128963319s" totalImagesPullingTime="7.578728371s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:20:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:20:27.128498257 +0000 UTC m=+10.684720533" watchObservedRunningTime="2026-10-30 19:20:27.128963319 +0000 UTC m=+10.685185596"
Oct 30 19:20:27.597962 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:27.597916    2574 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-nw4jx"
Oct 30 19:20:27.599468 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:27.599290    2574 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-nw4jx"
Oct 30 19:20:28.093127 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:28.092925    2574 generic.go:356] "Generic (PLEG): container finished" podID="1ceff922-50ab-4e0f-b113-03719eb40bc9" containerID="994387be7de962068f59d224b4f232811bb40cbf6fb5037937f314175f648464" exitCode=0
Oct 30 19:20:28.093575 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:28.093014    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hcmmq" event={"ID":"1ceff922-50ab-4e0f-b113-03719eb40bc9","Type":"ContainerDied","Data":"994387be7de962068f59d224b4f232811bb40cbf6fb5037937f314175f648464"}
Oct 30 19:20:28.094976 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:28.094950    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-f7tbh" event={"ID":"b41606bf-e8ca-4fc6-8805-b7461187d65e","Type":"ContainerStarted","Data":"f9a58a94b8908d2ab67818e3f51c2ec782cfb1d1a4e8a2c038e0d3409f3318b1"}
Oct 30 19:20:28.097444 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:28.097417    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch" event={"ID":"8161e358-9105-4d99-88c1-98c17db09674","Type":"ContainerStarted","Data":"56f7d317fd67ddccb5b8d6e19de2b862df13af2a0139f796a0fe24044c33d9a3"}
Oct 30 19:20:28.178513 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:28.178404    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch" podStartSLOduration=2.865538215 podStartE2EDuration="11.178386862s" totalImagesPullingTime="8.312848647s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:20:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:20:28.177777996 +0000 UTC m=+11.734000291" watchObservedRunningTime="2026-10-30 19:20:28.178386862 +0000 UTC m=+11.734609140"
Oct 30 19:20:28.214078 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:28.214012    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-f7tbh" podStartSLOduration=3.230057913 podStartE2EDuration="11.213991771s" totalImagesPullingTime="7.983933858s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:20:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:20:28.213123584 +0000 UTC m=+11.769345883" watchObservedRunningTime="2026-10-30 19:20:28.213991771 +0000 UTC m=+11.770214056"
Oct 30 19:20:28.949554 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:28.949460    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:20:28.949554 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:28.949542    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:20:28.949554 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:28.949539    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:20:28.949811 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:28.949649    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z7mhx" podUID="725d2cce-2a75-426f-8522-d1ddfbabe1b6"
Oct 30 19:20:28.950119 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:28.949983    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5j799" podUID="cd444ba1-edf1-49aa-a0cf-b863c93e60be"
Oct 30 19:20:28.950119 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:28.950097    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jk99" podUID="41ff5ee2-70a2-4baf-af4e-00e6575c2e99"
Oct 30 19:20:29.104098 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:29.104026    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-c9vlh" event={"ID":"57769216-dd32-4edc-8e73-b4873caa041d","Type":"ContainerStarted","Data":"52cebc16cc5da9d363fed47a35dc4398750f9bce7a2fb4c6d5a98e9bb26c9993"}
Oct 30 19:20:29.108568 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:29.108532    2574 generic.go:356] "Generic (PLEG): container finished" podID="1ceff922-50ab-4e0f-b113-03719eb40bc9" containerID="0c7400064cfbec0cd6dfefe6cf1c8790e09d5c9d8a836e99a283a0304494ae42" exitCode=0
Oct 30 19:20:29.108710 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:29.108642    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hcmmq" event={"ID":"1ceff922-50ab-4e0f-b113-03719eb40bc9","Type":"ContainerDied","Data":"0c7400064cfbec0cd6dfefe6cf1c8790e09d5c9d8a836e99a283a0304494ae42"}
Oct 30 19:20:29.140342 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:29.140283    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-c9vlh" podStartSLOduration=3.875846956 podStartE2EDuration="12.140264934s" totalImagesPullingTime="8.264417978s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:20:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:20:29.139932037 +0000 UTC m=+12.696154329" watchObservedRunningTime="2026-10-30 19:20:29.140264934 +0000 UTC m=+12.696487225"
Oct 30 19:20:30.114099 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:30.114068    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hcmmq" event={"ID":"1ceff922-50ab-4e0f-b113-03719eb40bc9","Type":"ContainerStarted","Data":"dc3f3e002810b5a0c835fb09a6c51df7ef21a21c130bbdfdf220fcc7fd2b52fe"}
Oct 30 19:20:30.115966 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:30.115927    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dvzwm" event={"ID":"a8635205-1f61-4a17-bc1f-7876c5536f9c","Type":"ContainerStarted","Data":"3e64b90af240980637a25308bae34d4b191507d81cc702d03e8416d821109dc1"}
Oct 30 19:20:30.211156 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:30.211087    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-dvzwm" podStartSLOduration=2.722762521 podStartE2EDuration="13.211071193s" totalImagesPullingTime="10.488308672s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:20:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:20:30.210391736 +0000 UTC m=+13.766614015" watchObservedRunningTime="2026-10-30 19:20:30.211071193 +0000 UTC m=+13.767293470"
Oct 30 19:20:30.949240 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:30.949205    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:20:30.949240 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:30.949251    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:20:30.949450 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:30.949205    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:20:30.949450 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:30.949334    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5j799" podUID="cd444ba1-edf1-49aa-a0cf-b863c93e60be"
Oct 30 19:20:30.949450 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:30.949405    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jk99" podUID="41ff5ee2-70a2-4baf-af4e-00e6575c2e99"
Oct 30 19:20:30.949607 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:30.949479    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z7mhx" podUID="725d2cce-2a75-426f-8522-d1ddfbabe1b6"
Oct 30 19:20:31.121390 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:31.121354    2574 generic.go:356] "Generic (PLEG): container finished" podID="1ceff922-50ab-4e0f-b113-03719eb40bc9" containerID="dc3f3e002810b5a0c835fb09a6c51df7ef21a21c130bbdfdf220fcc7fd2b52fe" exitCode=0
Oct 30 19:20:31.121913 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:31.121453    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hcmmq" event={"ID":"1ceff922-50ab-4e0f-b113-03719eb40bc9","Type":"ContainerDied","Data":"dc3f3e002810b5a0c835fb09a6c51df7ef21a21c130bbdfdf220fcc7fd2b52fe"}
Oct 30 19:20:32.948821 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:32.948622    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:20:32.949390 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:32.948620    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:20:32.949390 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:32.948989    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jk99" podUID="41ff5ee2-70a2-4baf-af4e-00e6575c2e99"
Oct 30 19:20:32.949390 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:32.949071    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5j799" podUID="cd444ba1-edf1-49aa-a0cf-b863c93e60be"
Oct 30 19:20:32.949390 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:32.948656    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:20:32.949390 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:32.949163    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z7mhx" podUID="725d2cce-2a75-426f-8522-d1ddfbabe1b6"
Oct 30 19:20:33.128367 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:33.128269    2574 generic.go:356] "Generic (PLEG): container finished" podID="1ceff922-50ab-4e0f-b113-03719eb40bc9" containerID="ec405270d7e6b82f7c8acb8a0d945abc44edbea3d17535efce7c8fe8363be98e" exitCode=0
Oct 30 19:20:33.128367 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:33.128325    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hcmmq" event={"ID":"1ceff922-50ab-4e0f-b113-03719eb40bc9","Type":"ContainerDied","Data":"ec405270d7e6b82f7c8acb8a0d945abc44edbea3d17535efce7c8fe8363be98e"}
Oct 30 19:20:34.134163 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:34.134135    2574 generic.go:356] "Generic (PLEG): container finished" podID="1ceff922-50ab-4e0f-b113-03719eb40bc9" containerID="5150e562ea857cb8be8b0f0c54355d38e28eebb95d2639aeaf7188f0b415a7c5" exitCode=0
Oct 30 19:20:34.134462 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:34.134175    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hcmmq" event={"ID":"1ceff922-50ab-4e0f-b113-03719eb40bc9","Type":"ContainerDied","Data":"5150e562ea857cb8be8b0f0c54355d38e28eebb95d2639aeaf7188f0b415a7c5"}
Oct 30 19:20:34.682702 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:34.682461    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41ff5ee2-70a2-4baf-af4e-00e6575c2e99-metrics-certs\") pod \"network-metrics-daemon-9jk99\" (UID: \"41ff5ee2-70a2-4baf-af4e-00e6575c2e99\") " pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:20:34.682883 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:34.682606    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:20:34.682883 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:34.682758    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41ff5ee2-70a2-4baf-af4e-00e6575c2e99-metrics-certs podName:41ff5ee2-70a2-4baf-af4e-00e6575c2e99 nodeName:}" failed. No retries permitted until 2026-10-30 19:20:50.682744021 +0000 UTC m=+34.238966275 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/41ff5ee2-70a2-4baf-af4e-00e6575c2e99-metrics-certs") pod "network-metrics-daemon-9jk99" (UID: "41ff5ee2-70a2-4baf-af4e-00e6575c2e99") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:20:34.682883 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:34.682699    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/725d2cce-2a75-426f-8522-d1ddfbabe1b6-original-pull-secret\") pod \"global-pull-secret-syncer-z7mhx\" (UID: \"725d2cce-2a75-426f-8522-d1ddfbabe1b6\") " pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:20:34.682883 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:34.682775    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 19:20:34.682883 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:34.682832    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/725d2cce-2a75-426f-8522-d1ddfbabe1b6-original-pull-secret podName:725d2cce-2a75-426f-8522-d1ddfbabe1b6 nodeName:}" failed. No retries permitted until 2026-10-30 19:20:50.682818241 +0000 UTC m=+34.239040496 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/725d2cce-2a75-426f-8522-d1ddfbabe1b6-original-pull-secret") pod "global-pull-secret-syncer-z7mhx" (UID: "725d2cce-2a75-426f-8522-d1ddfbabe1b6") : object "kube-system"/"original-pull-secret" not registered
Oct 30 19:20:34.783851 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:34.783813    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2xwk\" (UniqueName: \"kubernetes.io/projected/cd444ba1-edf1-49aa-a0cf-b863c93e60be-kube-api-access-c2xwk\") pod \"network-check-target-5j799\" (UID: \"cd444ba1-edf1-49aa-a0cf-b863c93e60be\") " pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:20:34.784034 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:34.783957    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 19:20:34.784034 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:34.783977    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 19:20:34.784034 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:34.783987    2574 projected.go:196] Error preparing data for projected volume kube-api-access-c2xwk for pod openshift-network-diagnostics/network-check-target-5j799: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:20:34.784034 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:34.784033    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd444ba1-edf1-49aa-a0cf-b863c93e60be-kube-api-access-c2xwk podName:cd444ba1-edf1-49aa-a0cf-b863c93e60be nodeName:}" failed. No retries permitted until 2026-10-30 19:20:50.784021801 +0000 UTC m=+34.340244070 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-c2xwk" (UniqueName: "kubernetes.io/projected/cd444ba1-edf1-49aa-a0cf-b863c93e60be-kube-api-access-c2xwk") pod "network-check-target-5j799" (UID: "cd444ba1-edf1-49aa-a0cf-b863c93e60be") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:20:34.949001 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:34.948924    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:20:34.949001 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:34.948949    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:20:34.949172 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:34.949020    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5j799" podUID="cd444ba1-edf1-49aa-a0cf-b863c93e60be"
Oct 30 19:20:34.949172 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:34.949058    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:20:34.949172 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:34.949143    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jk99" podUID="41ff5ee2-70a2-4baf-af4e-00e6575c2e99"
Oct 30 19:20:34.949310 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:34.949258    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z7mhx" podUID="725d2cce-2a75-426f-8522-d1ddfbabe1b6"
Oct 30 19:20:35.138919 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:35.138883    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8pv94" event={"ID":"23ec668e-b2fb-471f-a7c7-13095a3dcf7f","Type":"ContainerStarted","Data":"6aad9b7a74d1b2b5b656f3e792d9625ab165fb13d4d9ae8391b9dde0ae4d2636"}
Oct 30 19:20:35.139338 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:35.138966    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8pv94" event={"ID":"23ec668e-b2fb-471f-a7c7-13095a3dcf7f","Type":"ContainerStarted","Data":"dee304771ae813ac623a1619b1a76468c54cf8a09ebb5eac6a20abfdf2626256"}
Oct 30 19:20:35.139338 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:35.139013    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8pv94" event={"ID":"23ec668e-b2fb-471f-a7c7-13095a3dcf7f","Type":"ContainerStarted","Data":"2369439cc096fbce5f3b7cc3842f322739bf709ba54a211d4725c008526691b9"}
Oct 30 19:20:35.139338 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:35.139059    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8pv94" event={"ID":"23ec668e-b2fb-471f-a7c7-13095a3dcf7f","Type":"ContainerStarted","Data":"05cfcd0c796bae2e6b587f70cfa6b4f5e721554f0c35809ad253a3a89a37e56a"}
Oct 30 19:20:35.139338 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:35.139106    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8pv94" event={"ID":"23ec668e-b2fb-471f-a7c7-13095a3dcf7f","Type":"ContainerStarted","Data":"696cba4f0db28fc88ac885ad1807e277d8431c3a74b63e2cb67fbdc644f67b08"}
Oct 30 19:20:35.139338 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:35.139152    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8pv94" event={"ID":"23ec668e-b2fb-471f-a7c7-13095a3dcf7f","Type":"ContainerStarted","Data":"f32ad8d875b243f494bfdb40a7b993214348acbbbc51478ac34a9412e651cd77"}
Oct 30 19:20:35.144932 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:35.144903    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hcmmq" event={"ID":"1ceff922-50ab-4e0f-b113-03719eb40bc9","Type":"ContainerStarted","Data":"ac1e7a8ac441fb61396c3ddec5498e0bb447570f30ca870072cd1bea267fcc92"}
Oct 30 19:20:35.186112 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:35.186065    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-hcmmq" podStartSLOduration=7.798586232 podStartE2EDuration="18.186052886s" totalImagesPullingTime="10.387466654s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:20:17 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:20:35.185678097 +0000 UTC m=+18.741900374" watchObservedRunningTime="2026-10-30 19:20:35.186052886 +0000 UTC m=+18.742275163"
Oct 30 19:20:36.090792 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:36.090757    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-nw4jx"
Oct 30 19:20:36.091048 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:36.090922    2574 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 19:20:36.091524 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:36.091499    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-nw4jx"
Oct 30 19:20:36.950275 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:36.950201    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:20:36.950796 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:36.950304    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:20:36.950796 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:36.950317    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z7mhx" podUID="725d2cce-2a75-426f-8522-d1ddfbabe1b6"
Oct 30 19:20:36.950796 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:36.950340    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:20:36.950796 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:36.950408    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jk99" podUID="41ff5ee2-70a2-4baf-af4e-00e6575c2e99"
Oct 30 19:20:36.950796 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:36.950466    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5j799" podUID="cd444ba1-edf1-49aa-a0cf-b863c93e60be"
Oct 30 19:20:37.152588 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:37.152541    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8pv94" event={"ID":"23ec668e-b2fb-471f-a7c7-13095a3dcf7f","Type":"ContainerStarted","Data":"6bc25e99ed0c5b20cf695a5d90de8bffba1da9429276a40ca0ebf34c009cc950"}
Oct 30 19:20:38.949294 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:38.949201    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:20:38.949294 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:38.949267    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:20:38.950085 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:38.949383    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5j799" podUID="cd444ba1-edf1-49aa-a0cf-b863c93e60be"
Oct 30 19:20:38.950085 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:38.949414    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:20:38.950085 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:38.949486    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jk99" podUID="41ff5ee2-70a2-4baf-af4e-00e6575c2e99"
Oct 30 19:20:38.950085 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:38.949574    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z7mhx" podUID="725d2cce-2a75-426f-8522-d1ddfbabe1b6"
Oct 30 19:20:39.159928 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:39.159888    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8pv94" event={"ID":"23ec668e-b2fb-471f-a7c7-13095a3dcf7f","Type":"ContainerStarted","Data":"78bd66757b728cb6507cb838cc91d9ff16abe5fd392986bcad91f0d27f33b9a2"}
Oct 30 19:20:39.160189 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:39.160169    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:39.176543 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:39.176381    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:39.246800 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:39.246694    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-8pv94" podStartSLOduration=7.344018495 podStartE2EDuration="22.246681924s" totalImagesPullingTime="14.902663429s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:20:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:20:39.235223646 +0000 UTC m=+22.791445923" watchObservedRunningTime="2026-10-30 19:20:39.246681924 +0000 UTC m=+22.802904201"
Oct 30 19:20:40.161695 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:40.161661    2574 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 19:20:40.162302 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:40.162208    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:40.180628 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:40.178642    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:40.949155 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:40.949123    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:20:40.949322 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:40.949164    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:20:40.949322 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:40.949234    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5j799" podUID="cd444ba1-edf1-49aa-a0cf-b863c93e60be"
Oct 30 19:20:40.949322 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:40.949320    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:20:40.949478 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:40.949429    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jk99" podUID="41ff5ee2-70a2-4baf-af4e-00e6575c2e99"
Oct 30 19:20:40.949548 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:40.949525    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z7mhx" podUID="725d2cce-2a75-426f-8522-d1ddfbabe1b6"
Oct 30 19:20:41.164170 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:41.164145    2574 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 19:20:42.165937 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:42.165909    2574 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 19:20:42.949638 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:42.949608    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:20:42.949638 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:42.949624    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:20:42.949834 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:42.949722    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z7mhx" podUID="725d2cce-2a75-426f-8522-d1ddfbabe1b6"
Oct 30 19:20:42.949834 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:42.949764    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5j799" podUID="cd444ba1-edf1-49aa-a0cf-b863c93e60be"
Oct 30 19:20:42.949834 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:42.949804    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:20:42.949998 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:42.949922    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jk99" podUID="41ff5ee2-70a2-4baf-af4e-00e6575c2e99"
Oct 30 19:20:43.371749 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:43.371666    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-f7tbh_b41606bf-e8ca-4fc6-8805-b7461187d65e/dns-node-resolver/0.log"
Oct 30 19:20:44.527727 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:44.527698    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tgt5g_ba4ee546-c9c1-4573-b11f-6ba0621972d1/node-ca/0.log"
Oct 30 19:20:44.532853 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:44.532833    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:20:44.533112 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:44.533091    2574 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 19:20:44.543636 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:44.543595    2574 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-8pv94" podUID="23ec668e-b2fb-471f-a7c7-13095a3dcf7f" containerName="ovnkube-controller" probeResult="failure" output=""
Oct 30 19:20:44.552637 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:44.552595    2574 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-8pv94" podUID="23ec668e-b2fb-471f-a7c7-13095a3dcf7f" containerName="ovnkube-controller" probeResult="failure" output=""
Oct 30 19:20:44.948814 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:44.948726    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:20:44.948814 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:44.948765    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:20:44.949032 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:44.948828    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5j799" podUID="cd444ba1-edf1-49aa-a0cf-b863c93e60be"
Oct 30 19:20:44.949032 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:44.948916    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:20:44.949032 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:44.948999    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jk99" podUID="41ff5ee2-70a2-4baf-af4e-00e6575c2e99"
Oct 30 19:20:44.949128 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:44.949070    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z7mhx" podUID="725d2cce-2a75-426f-8522-d1ddfbabe1b6"
Oct 30 19:20:46.949413 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:46.949384    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:20:46.949814 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:46.949512    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:20:46.949814 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:46.949528    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5j799" podUID="cd444ba1-edf1-49aa-a0cf-b863c93e60be"
Oct 30 19:20:46.949814 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:46.949614    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jk99" podUID="41ff5ee2-70a2-4baf-af4e-00e6575c2e99"
Oct 30 19:20:46.949814 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:46.949656    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:20:46.949814 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:46.949724    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z7mhx" podUID="725d2cce-2a75-426f-8522-d1ddfbabe1b6"
Oct 30 19:20:47.168238 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:47.168197    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nw4jx_2c840c6b-6992-4f29-9023-a0ad99fdc543/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:47.324116 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:47.324017    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nw4jx_2c840c6b-6992-4f29-9023-a0ad99fdc543/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:47.506380 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:47.506327    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nw4jx_2c840c6b-6992-4f29-9023-a0ad99fdc543/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:47.721162 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:47.721086    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nw4jx_2c840c6b-6992-4f29-9023-a0ad99fdc543/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:48.949311 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:48.949271    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:20:48.949702 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:48.949284    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:20:48.949702 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:48.949399    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z7mhx" podUID="725d2cce-2a75-426f-8522-d1ddfbabe1b6"
Oct 30 19:20:48.949702 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:48.949291    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:20:48.949702 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:48.949467    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jk99" podUID="41ff5ee2-70a2-4baf-af4e-00e6575c2e99"
Oct 30 19:20:48.949702 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:48.949538    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5j799" podUID="cd444ba1-edf1-49aa-a0cf-b863c93e60be"
Oct 30 19:20:50.685905 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:50.685873    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/725d2cce-2a75-426f-8522-d1ddfbabe1b6-original-pull-secret\") pod \"global-pull-secret-syncer-z7mhx\" (UID: \"725d2cce-2a75-426f-8522-d1ddfbabe1b6\") " pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:20:50.686356 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:50.685922    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41ff5ee2-70a2-4baf-af4e-00e6575c2e99-metrics-certs\") pod \"network-metrics-daemon-9jk99\" (UID: \"41ff5ee2-70a2-4baf-af4e-00e6575c2e99\") " pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:20:50.686356 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:50.685999    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 19:20:50.686356 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:50.686032    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:20:50.686356 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:50.686057    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/725d2cce-2a75-426f-8522-d1ddfbabe1b6-original-pull-secret podName:725d2cce-2a75-426f-8522-d1ddfbabe1b6 nodeName:}" failed. No retries permitted until 2026-10-30 19:21:22.6860429 +0000 UTC m=+66.242265159 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/725d2cce-2a75-426f-8522-d1ddfbabe1b6-original-pull-secret") pod "global-pull-secret-syncer-z7mhx" (UID: "725d2cce-2a75-426f-8522-d1ddfbabe1b6") : object "kube-system"/"original-pull-secret" not registered
Oct 30 19:20:50.686356 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:50.686086    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41ff5ee2-70a2-4baf-af4e-00e6575c2e99-metrics-certs podName:41ff5ee2-70a2-4baf-af4e-00e6575c2e99 nodeName:}" failed. No retries permitted until 2026-10-30 19:21:22.686068742 +0000 UTC m=+66.242290997 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/41ff5ee2-70a2-4baf-af4e-00e6575c2e99-metrics-certs") pod "network-metrics-daemon-9jk99" (UID: "41ff5ee2-70a2-4baf-af4e-00e6575c2e99") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:20:50.786566 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:50.786535    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2xwk\" (UniqueName: \"kubernetes.io/projected/cd444ba1-edf1-49aa-a0cf-b863c93e60be-kube-api-access-c2xwk\") pod \"network-check-target-5j799\" (UID: \"cd444ba1-edf1-49aa-a0cf-b863c93e60be\") " pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:20:50.786721 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:50.786649    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 19:20:50.786721 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:50.786662    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 19:20:50.786721 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:50.786670    2574 projected.go:196] Error preparing data for projected volume kube-api-access-c2xwk for pod openshift-network-diagnostics/network-check-target-5j799: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:20:50.786721 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:50.786717    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd444ba1-edf1-49aa-a0cf-b863c93e60be-kube-api-access-c2xwk podName:cd444ba1-edf1-49aa-a0cf-b863c93e60be nodeName:}" failed. No retries permitted until 2026-10-30 19:21:22.786703648 +0000 UTC m=+66.342925921 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-c2xwk" (UniqueName: "kubernetes.io/projected/cd444ba1-edf1-49aa-a0cf-b863c93e60be-kube-api-access-c2xwk") pod "network-check-target-5j799" (UID: "cd444ba1-edf1-49aa-a0cf-b863c93e60be") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:20:50.951927 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:50.951832    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:20:50.952083 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:50.951972    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:20:50.952083 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:50.951979    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jk99" podUID="41ff5ee2-70a2-4baf-af4e-00e6575c2e99"
Oct 30 19:20:50.952083 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:50.951996    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:20:50.952083 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:50.952062    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5j799" podUID="cd444ba1-edf1-49aa-a0cf-b863c93e60be"
Oct 30 19:20:50.952271 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:50.952117    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z7mhx" podUID="725d2cce-2a75-426f-8522-d1ddfbabe1b6"
Oct 30 19:20:52.951530 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:52.951501    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:20:52.951530 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:52.951521    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:20:52.951980 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:52.951598    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z7mhx" podUID="725d2cce-2a75-426f-8522-d1ddfbabe1b6"
Oct 30 19:20:52.951980 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:52.951644    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:20:52.951980 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:52.951712    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5j799" podUID="cd444ba1-edf1-49aa-a0cf-b863c93e60be"
Oct 30 19:20:52.951980 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:52.951820    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jk99" podUID="41ff5ee2-70a2-4baf-af4e-00e6575c2e99"
Oct 30 19:20:54.949481 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:54.949441    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:20:54.949481 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:54.949480    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:20:54.950046 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:54.949445    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:20:54.950046 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:54.949555    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z7mhx" podUID="725d2cce-2a75-426f-8522-d1ddfbabe1b6"
Oct 30 19:20:54.950046 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:54.949618    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5j799" podUID="cd444ba1-edf1-49aa-a0cf-b863c93e60be"
Oct 30 19:20:54.950046 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:54.949742    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jk99" podUID="41ff5ee2-70a2-4baf-af4e-00e6575c2e99"
Oct 30 19:20:56.951305 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:56.951277    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:20:56.951850 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:56.951334    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:20:56.951850 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:56.951415    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:20:56.951850 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:56.951438    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jk99" podUID="41ff5ee2-70a2-4baf-af4e-00e6575c2e99"
Oct 30 19:20:56.951850 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:56.951479    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z7mhx" podUID="725d2cce-2a75-426f-8522-d1ddfbabe1b6"
Oct 30 19:20:56.951850 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:56.951547    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5j799" podUID="cd444ba1-edf1-49aa-a0cf-b863c93e60be"
Oct 30 19:20:58.949039 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:58.949005    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:20:58.949039 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:58.949041    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:20:58.949616 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:20:58.949011    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:20:58.949616 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:58.949111    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z7mhx" podUID="725d2cce-2a75-426f-8522-d1ddfbabe1b6"
Oct 30 19:20:58.949616 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:58.949203    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5j799" podUID="cd444ba1-edf1-49aa-a0cf-b863c93e60be"
Oct 30 19:20:58.949616 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:20:58.949292    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jk99" podUID="41ff5ee2-70a2-4baf-af4e-00e6575c2e99"
Oct 30 19:21:00.644149 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:00.644119    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-z7mhx"]
Oct 30 19:21:00.644602 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:00.644289    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:21:00.644602 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:21:00.644393    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z7mhx" podUID="725d2cce-2a75-426f-8522-d1ddfbabe1b6"
Oct 30 19:21:00.647142 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:00.647089    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9jk99"]
Oct 30 19:21:00.647261 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:00.647240    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:21:00.647385 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:21:00.647360    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jk99" podUID="41ff5ee2-70a2-4baf-af4e-00e6575c2e99"
Oct 30 19:21:00.647921 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:00.647900    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5j799"]
Oct 30 19:21:00.648008 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:00.647994    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:21:00.648097 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:21:00.648080    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5j799" podUID="cd444ba1-edf1-49aa-a0cf-b863c93e60be"
Oct 30 19:21:01.949092 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:01.948908    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:21:01.949457 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:01.948932    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:21:01.949457 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:21:01.949169    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5j799" podUID="cd444ba1-edf1-49aa-a0cf-b863c93e60be"
Oct 30 19:21:01.949457 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:21:01.949251    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jk99" podUID="41ff5ee2-70a2-4baf-af4e-00e6575c2e99"
Oct 30 19:21:02.949442 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:02.949407    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:21:02.949847 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:21:02.949556    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z7mhx" podUID="725d2cce-2a75-426f-8522-d1ddfbabe1b6"
Oct 30 19:21:03.949166 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:03.949130    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:21:03.949166 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:03.949162    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:21:03.949367 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:21:03.949264    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jk99" podUID="41ff5ee2-70a2-4baf-af4e-00e6575c2e99"
Oct 30 19:21:03.949407 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:21:03.949379    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5j799" podUID="cd444ba1-edf1-49aa-a0cf-b863c93e60be"
Oct 30 19:21:04.949640 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:04.949569    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:21:04.950009 ip-10-0-132-152 kubenswrapper[2574]: E1030 19:21:04.949669    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z7mhx" podUID="725d2cce-2a75-426f-8522-d1ddfbabe1b6"
Oct 30 19:21:05.610145 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.610121    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-152.ec2.internal" event="NodeReady"
Oct 30 19:21:05.610289 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.610244    2574 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 19:21:05.667244 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.667211    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vn5fp"]
Oct 30 19:21:05.669098 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.669066    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vn5fp"
Oct 30 19:21:05.671376 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.671355    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-p8nq5"]
Oct 30 19:21:05.673121 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.673092    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-p8nq5"
Oct 30 19:21:05.683386 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.683364    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-m2qt2\""
Oct 30 19:21:05.683386 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.683381    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 19:21:05.683507 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.683401    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 19:21:05.697559 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.697534    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-q2cc6"]
Oct 30 19:21:05.699445 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.699424    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vn5fp"]
Oct 30 19:21:05.699564 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.699543    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-q2cc6"
Oct 30 19:21:05.699791 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.699772    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-p8nq5"]
Oct 30 19:21:05.723005 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.722978    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-q2cc6"]
Oct 30 19:21:05.781322 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.781293    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k9b2\" (UniqueName: \"kubernetes.io/projected/9a1d2cec-f539-4106-a185-3134a3df747c-kube-api-access-8k9b2\") pod \"insights-runtime-extractor-p8nq5\" (UID: \"9a1d2cec-f539-4106-a185-3134a3df747c\") " pod="openshift-insights/insights-runtime-extractor-p8nq5"
Oct 30 19:21:05.781510 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.781337    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9a1d2cec-f539-4106-a185-3134a3df747c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p8nq5\" (UID: \"9a1d2cec-f539-4106-a185-3134a3df747c\") " pod="openshift-insights/insights-runtime-extractor-p8nq5"
Oct 30 19:21:05.781510 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.781363    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b70ef66-ed5b-4917-8bca-36757fee37e3-metrics-tls\") pod \"dns-default-vn5fp\" (UID: \"4b70ef66-ed5b-4917-8bca-36757fee37e3\") " pod="openshift-dns/dns-default-vn5fp"
Oct 30 19:21:05.781510 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.781403    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9a1d2cec-f539-4106-a185-3134a3df747c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p8nq5\" (UID: \"9a1d2cec-f539-4106-a185-3134a3df747c\") " pod="openshift-insights/insights-runtime-extractor-p8nq5"
Oct 30 19:21:05.781510 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.781485    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a1d2cec-f539-4106-a185-3134a3df747c-data-volume\") pod \"insights-runtime-extractor-p8nq5\" (UID: \"9a1d2cec-f539-4106-a185-3134a3df747c\") " pod="openshift-insights/insights-runtime-extractor-p8nq5"
Oct 30 19:21:05.781654 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.781520    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9a1d2cec-f539-4106-a185-3134a3df747c-crio-socket\") pod \"insights-runtime-extractor-p8nq5\" (UID: \"9a1d2cec-f539-4106-a185-3134a3df747c\") " pod="openshift-insights/insights-runtime-extractor-p8nq5"
Oct 30 19:21:05.781654 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.781538    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b70ef66-ed5b-4917-8bca-36757fee37e3-tmp-dir\") pod \"dns-default-vn5fp\" (UID: \"4b70ef66-ed5b-4917-8bca-36757fee37e3\") " pod="openshift-dns/dns-default-vn5fp"
Oct 30 19:21:05.781654 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.781553    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvcz2\" (UniqueName: \"kubernetes.io/projected/4b70ef66-ed5b-4917-8bca-36757fee37e3-kube-api-access-mvcz2\") pod \"dns-default-vn5fp\" (UID: \"4b70ef66-ed5b-4917-8bca-36757fee37e3\") " pod="openshift-dns/dns-default-vn5fp"
Oct 30 19:21:05.781654 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.781574    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4b70ef66-ed5b-4917-8bca-36757fee37e3-config-volume\") pod \"dns-default-vn5fp\" (UID: \"4b70ef66-ed5b-4917-8bca-36757fee37e3\") " pod="openshift-dns/dns-default-vn5fp"
Oct 30 19:21:05.783563 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.783541    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 19:21:05.783714 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.783589    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 19:21:05.783714 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.783602    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 19:21:05.783714 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.783606    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 19:21:05.783714 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.783663    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 19:21:05.783714 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.783672    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 19:21:05.783892 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.783756    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qmnn8\""
Oct 30 19:21:05.783892 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.783806    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 19:21:05.783892 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.783888    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-lkdbb\""
Oct 30 19:21:05.882508 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.882414    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k9b2\" (UniqueName: \"kubernetes.io/projected/9a1d2cec-f539-4106-a185-3134a3df747c-kube-api-access-8k9b2\") pod \"insights-runtime-extractor-p8nq5\" (UID: \"9a1d2cec-f539-4106-a185-3134a3df747c\") " pod="openshift-insights/insights-runtime-extractor-p8nq5"
Oct 30 19:21:05.882508 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.882468    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9a1d2cec-f539-4106-a185-3134a3df747c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p8nq5\" (UID: \"9a1d2cec-f539-4106-a185-3134a3df747c\") " pod="openshift-insights/insights-runtime-extractor-p8nq5"
Oct 30 19:21:05.882508 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.882493    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c59e8b44-50b7-4a50-af6a-1bb9a7547a56-cert\") pod \"ingress-canary-q2cc6\" (UID: \"c59e8b44-50b7-4a50-af6a-1bb9a7547a56\") " pod="openshift-ingress-canary/ingress-canary-q2cc6"
Oct 30 19:21:05.882753 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.882545    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5c8j\" (UniqueName: \"kubernetes.io/projected/c59e8b44-50b7-4a50-af6a-1bb9a7547a56-kube-api-access-t5c8j\") pod \"ingress-canary-q2cc6\" (UID: \"c59e8b44-50b7-4a50-af6a-1bb9a7547a56\") " pod="openshift-ingress-canary/ingress-canary-q2cc6"
Oct 30 19:21:05.882753 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.882586    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b70ef66-ed5b-4917-8bca-36757fee37e3-metrics-tls\") pod \"dns-default-vn5fp\" (UID: \"4b70ef66-ed5b-4917-8bca-36757fee37e3\") " pod="openshift-dns/dns-default-vn5fp"
Oct 30 19:21:05.882753 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.882608    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9a1d2cec-f539-4106-a185-3134a3df747c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p8nq5\" (UID: \"9a1d2cec-f539-4106-a185-3134a3df747c\") " pod="openshift-insights/insights-runtime-extractor-p8nq5"
Oct 30 19:21:05.882753 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.882635    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a1d2cec-f539-4106-a185-3134a3df747c-data-volume\") pod \"insights-runtime-extractor-p8nq5\" (UID: \"9a1d2cec-f539-4106-a185-3134a3df747c\") " pod="openshift-insights/insights-runtime-extractor-p8nq5"
Oct 30 19:21:05.882753 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.882663    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9a1d2cec-f539-4106-a185-3134a3df747c-crio-socket\") pod \"insights-runtime-extractor-p8nq5\" (UID: \"9a1d2cec-f539-4106-a185-3134a3df747c\") " pod="openshift-insights/insights-runtime-extractor-p8nq5"
Oct 30 19:21:05.882753 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.882731    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b70ef66-ed5b-4917-8bca-36757fee37e3-tmp-dir\") pod \"dns-default-vn5fp\" (UID: \"4b70ef66-ed5b-4917-8bca-36757fee37e3\") " pod="openshift-dns/dns-default-vn5fp"
Oct 30 19:21:05.883064 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.882755    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvcz2\" (UniqueName: \"kubernetes.io/projected/4b70ef66-ed5b-4917-8bca-36757fee37e3-kube-api-access-mvcz2\") pod \"dns-default-vn5fp\" (UID: \"4b70ef66-ed5b-4917-8bca-36757fee37e3\") " pod="openshift-dns/dns-default-vn5fp"
Oct 30 19:21:05.883064 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.882793    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4b70ef66-ed5b-4917-8bca-36757fee37e3-config-volume\") pod \"dns-default-vn5fp\" (UID: \"4b70ef66-ed5b-4917-8bca-36757fee37e3\") " pod="openshift-dns/dns-default-vn5fp"
Oct 30 19:21:05.883170 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.883062    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a1d2cec-f539-4106-a185-3134a3df747c-data-volume\") pod \"insights-runtime-extractor-p8nq5\" (UID: \"9a1d2cec-f539-4106-a185-3134a3df747c\") " pod="openshift-insights/insights-runtime-extractor-p8nq5"
Oct 30 19:21:05.883170 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.883072    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b70ef66-ed5b-4917-8bca-36757fee37e3-tmp-dir\") pod \"dns-default-vn5fp\" (UID: \"4b70ef66-ed5b-4917-8bca-36757fee37e3\") " pod="openshift-dns/dns-default-vn5fp"
Oct 30 19:21:05.883258 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.883178    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9a1d2cec-f539-4106-a185-3134a3df747c-crio-socket\") pod \"insights-runtime-extractor-p8nq5\" (UID: \"9a1d2cec-f539-4106-a185-3134a3df747c\") " pod="openshift-insights/insights-runtime-extractor-p8nq5"
Oct 30 19:21:05.883300 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.883283    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9a1d2cec-f539-4106-a185-3134a3df747c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p8nq5\" (UID: \"9a1d2cec-f539-4106-a185-3134a3df747c\") " pod="openshift-insights/insights-runtime-extractor-p8nq5"
Oct 30 19:21:05.883353 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.883336    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4b70ef66-ed5b-4917-8bca-36757fee37e3-config-volume\") pod \"dns-default-vn5fp\" (UID: \"4b70ef66-ed5b-4917-8bca-36757fee37e3\") " pod="openshift-dns/dns-default-vn5fp"
Oct 30 19:21:05.886639 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.886614    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b70ef66-ed5b-4917-8bca-36757fee37e3-metrics-tls\") pod \"dns-default-vn5fp\" (UID: \"4b70ef66-ed5b-4917-8bca-36757fee37e3\") " pod="openshift-dns/dns-default-vn5fp"
Oct 30 19:21:05.886734 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.886676    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9a1d2cec-f539-4106-a185-3134a3df747c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p8nq5\" (UID: \"9a1d2cec-f539-4106-a185-3134a3df747c\") " pod="openshift-insights/insights-runtime-extractor-p8nq5"
Oct 30 19:21:05.900425 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.900393    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvcz2\" (UniqueName: \"kubernetes.io/projected/4b70ef66-ed5b-4917-8bca-36757fee37e3-kube-api-access-mvcz2\") pod \"dns-default-vn5fp\" (UID: \"4b70ef66-ed5b-4917-8bca-36757fee37e3\") " pod="openshift-dns/dns-default-vn5fp"
Oct 30 19:21:05.900714 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.900699    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k9b2\" (UniqueName: \"kubernetes.io/projected/9a1d2cec-f539-4106-a185-3134a3df747c-kube-api-access-8k9b2\") pod \"insights-runtime-extractor-p8nq5\" (UID: \"9a1d2cec-f539-4106-a185-3134a3df747c\") " pod="openshift-insights/insights-runtime-extractor-p8nq5"
Oct 30 19:21:05.949027 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.948994    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:21:05.949205 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.949189    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:21:05.963086 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.963064    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 19:21:05.963374 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.963091    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 19:21:05.963374 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.963137    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sl59z\""
Oct 30 19:21:05.963374 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.963146    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 19:21:05.963374 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.963138    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-77hs9\""
Oct 30 19:21:05.981348 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.981328    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vn5fp"
Oct 30 19:21:05.983139 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.983121    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c59e8b44-50b7-4a50-af6a-1bb9a7547a56-cert\") pod \"ingress-canary-q2cc6\" (UID: \"c59e8b44-50b7-4a50-af6a-1bb9a7547a56\") " pod="openshift-ingress-canary/ingress-canary-q2cc6"
Oct 30 19:21:05.983176 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.983155    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5c8j\" (UniqueName: \"kubernetes.io/projected/c59e8b44-50b7-4a50-af6a-1bb9a7547a56-kube-api-access-t5c8j\") pod \"ingress-canary-q2cc6\" (UID: \"c59e8b44-50b7-4a50-af6a-1bb9a7547a56\") " pod="openshift-ingress-canary/ingress-canary-q2cc6"
Oct 30 19:21:05.985769 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.985746    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c59e8b44-50b7-4a50-af6a-1bb9a7547a56-cert\") pod \"ingress-canary-q2cc6\" (UID: \"c59e8b44-50b7-4a50-af6a-1bb9a7547a56\") " pod="openshift-ingress-canary/ingress-canary-q2cc6"
Oct 30 19:21:05.987454 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:05.987439    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-p8nq5"
Oct 30 19:21:06.002664 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:06.002643    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5c8j\" (UniqueName: \"kubernetes.io/projected/c59e8b44-50b7-4a50-af6a-1bb9a7547a56-kube-api-access-t5c8j\") pod \"ingress-canary-q2cc6\" (UID: \"c59e8b44-50b7-4a50-af6a-1bb9a7547a56\") " pod="openshift-ingress-canary/ingress-canary-q2cc6"
Oct 30 19:21:06.010397 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:06.010359    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-q2cc6"
Oct 30 19:21:06.294137 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:06.292204    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vn5fp"]
Oct 30 19:21:06.300479 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:21:06.300452    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b70ef66_ed5b_4917_8bca_36757fee37e3.slice/crio-2d7437ffa1bf640b1e1e7c6bf743f745578a07f4ca63d213d6bf15dc27281aea WatchSource:0}: Error finding container 2d7437ffa1bf640b1e1e7c6bf743f745578a07f4ca63d213d6bf15dc27281aea: Status 404 returned error can't find the container with id 2d7437ffa1bf640b1e1e7c6bf743f745578a07f4ca63d213d6bf15dc27281aea
Oct 30 19:21:06.305911 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:06.305888    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-p8nq5"]
Oct 30 19:21:06.311613 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:21:06.311591    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a1d2cec_f539_4106_a185_3134a3df747c.slice/crio-c37476d3d2fccb3f1018e0edaaeacc5cacdba55633f63c064597d568e0345fa6 WatchSource:0}: Error finding container c37476d3d2fccb3f1018e0edaaeacc5cacdba55633f63c064597d568e0345fa6: Status 404 returned error can't find the container with id c37476d3d2fccb3f1018e0edaaeacc5cacdba55633f63c064597d568e0345fa6
Oct 30 19:21:06.329032 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:06.329007    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-q2cc6"]
Oct 30 19:21:06.337803 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:21:06.337780    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc59e8b44_50b7_4a50_af6a_1bb9a7547a56.slice/crio-3e2295a83987368051b0ecf5c6b265d334ed32ee0f27fd4214a29d995ad7fd9f WatchSource:0}: Error finding container 3e2295a83987368051b0ecf5c6b265d334ed32ee0f27fd4214a29d995ad7fd9f: Status 404 returned error can't find the container with id 3e2295a83987368051b0ecf5c6b265d334ed32ee0f27fd4214a29d995ad7fd9f
Oct 30 19:21:06.951841 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:06.951810    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:21:06.977234 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:06.977200    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 19:21:06.977576 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:06.977311    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-65nsn\""
Oct 30 19:21:07.235406 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.235314    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-q2cc6" event={"ID":"c59e8b44-50b7-4a50-af6a-1bb9a7547a56","Type":"ContainerStarted","Data":"3e2295a83987368051b0ecf5c6b265d334ed32ee0f27fd4214a29d995ad7fd9f"}
Oct 30 19:21:07.238458 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.238428    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p8nq5" event={"ID":"9a1d2cec-f539-4106-a185-3134a3df747c","Type":"ContainerStarted","Data":"cbfc63f291de23a387781f2d944abdcf8379ffcb9e556a893e161a8ed6b14159"}
Oct 30 19:21:07.238584 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.238492    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p8nq5" event={"ID":"9a1d2cec-f539-4106-a185-3134a3df747c","Type":"ContainerStarted","Data":"53333c3ac0e27af4720cc9e2b6019f5bad1e0cb95e3e7e443ef212b8b7887e7d"}
Oct 30 19:21:07.238584 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.238526    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p8nq5" event={"ID":"9a1d2cec-f539-4106-a185-3134a3df747c","Type":"ContainerStarted","Data":"c37476d3d2fccb3f1018e0edaaeacc5cacdba55633f63c064597d568e0345fa6"}
Oct 30 19:21:07.240756 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.240727    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vn5fp" event={"ID":"4b70ef66-ed5b-4917-8bca-36757fee37e3","Type":"ContainerStarted","Data":"2d7437ffa1bf640b1e1e7c6bf743f745578a07f4ca63d213d6bf15dc27281aea"}
Oct 30 19:21:07.315558 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.315522    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d79db5f98-fmng5"]
Oct 30 19:21:07.317652 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.317628    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:07.338408 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.338382    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 19:21:07.338739 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.338645    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 30 19:21:07.338739 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.338654    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 19:21:07.338739 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.338676    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 30 19:21:07.338739 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.338681    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 30 19:21:07.338739 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.338684    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 30 19:21:07.338739 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.338716    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-h5vgg\""
Oct 30 19:21:07.339076 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.338795    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 30 19:21:07.346892 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.345914    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d79db5f98-fmng5"]
Oct 30 19:21:07.495442 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.495364    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-service-ca\") pod \"console-5d79db5f98-fmng5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") " pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:07.495442 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.495418    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-console-serving-cert\") pod \"console-5d79db5f98-fmng5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") " pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:07.495660 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.495511    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-tmp\") pod \"console-5d79db5f98-fmng5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") " pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:07.495660 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.495581    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-console-config\") pod \"console-5d79db5f98-fmng5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") " pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:07.495660 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.495605    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6klk6\" (UniqueName: \"kubernetes.io/projected/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-kube-api-access-6klk6\") pod \"console-5d79db5f98-fmng5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") " pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:07.495660 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.495644    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-oauth-serving-cert\") pod \"console-5d79db5f98-fmng5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") " pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:07.495848 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.495808    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-console-oauth-config\") pod \"console-5d79db5f98-fmng5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") " pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:07.596783 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.596747    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-oauth-serving-cert\") pod \"console-5d79db5f98-fmng5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") " pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:07.596976 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.596805    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-console-oauth-config\") pod \"console-5d79db5f98-fmng5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") " pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:07.596976 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.596837    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-service-ca\") pod \"console-5d79db5f98-fmng5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") " pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:07.596976 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.596887    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-console-serving-cert\") pod \"console-5d79db5f98-fmng5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") " pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:07.596976 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.596910    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-tmp\") pod \"console-5d79db5f98-fmng5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") " pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:07.597165 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.597086    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-console-config\") pod \"console-5d79db5f98-fmng5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") " pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:07.597165 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.597126    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6klk6\" (UniqueName: \"kubernetes.io/projected/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-kube-api-access-6klk6\") pod \"console-5d79db5f98-fmng5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") " pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:07.597404 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.597379    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-tmp\") pod \"console-5d79db5f98-fmng5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") " pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:07.597656 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.597626    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-oauth-serving-cert\") pod \"console-5d79db5f98-fmng5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") " pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:07.597656 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.597652    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-service-ca\") pod \"console-5d79db5f98-fmng5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") " pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:07.598005 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.597924    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-console-config\") pod \"console-5d79db5f98-fmng5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") " pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:07.599743 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.599721    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-console-serving-cert\") pod \"console-5d79db5f98-fmng5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") " pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:07.600433 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.600411    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-console-oauth-config\") pod \"console-5d79db5f98-fmng5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") " pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:07.619663 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.619637    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6klk6\" (UniqueName: \"kubernetes.io/projected/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-kube-api-access-6klk6\") pod \"console-5d79db5f98-fmng5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") " pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:07.630941 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.630915    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:07.941102 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:07.941069    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d79db5f98-fmng5"]
Oct 30 19:21:07.949123 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:21:07.949088    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44b9fc01_6837_46cd_bf01_40cb8fcb90f5.slice/crio-9f818a927ce3a7d5f3fe85edc4285eb3f0932cc7b6bfc950eec0956ed2e70d7f WatchSource:0}: Error finding container 9f818a927ce3a7d5f3fe85edc4285eb3f0932cc7b6bfc950eec0956ed2e70d7f: Status 404 returned error can't find the container with id 9f818a927ce3a7d5f3fe85edc4285eb3f0932cc7b6bfc950eec0956ed2e70d7f
Oct 30 19:21:08.245037 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.244946    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d79db5f98-fmng5" event={"ID":"44b9fc01-6837-46cd-bf01-40cb8fcb90f5","Type":"ContainerStarted","Data":"9f818a927ce3a7d5f3fe85edc4285eb3f0932cc7b6bfc950eec0956ed2e70d7f"}
Oct 30 19:21:08.564022 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.563996    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-8k92x"]
Oct 30 19:21:08.566325 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.566302    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8k92x"
Oct 30 19:21:08.592226 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.592111    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 19:21:08.599164 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.598948    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 19:21:08.599164 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.599019    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 30 19:21:08.599164 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.599057    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 30 19:21:08.599164 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.599130    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-mbz98\""
Oct 30 19:21:08.599164 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.599132    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 19:21:08.609842 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.608577    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh"]
Oct 30 19:21:08.611764 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.610988    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-8k92x"]
Oct 30 19:21:08.611764 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.611131    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh"
Oct 30 19:21:08.617974 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.616205    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-jqs6k"]
Oct 30 19:21:08.624930 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.624403    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:08.654315 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.653612    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Oct 30 19:21:08.654315 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.654016    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 19:21:08.654315 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.654052    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Oct 30 19:21:08.654315 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.654152    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gqsl5\""
Oct 30 19:21:08.654315 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.654274    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 19:21:08.654994 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.654946    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-2nv6s\""
Oct 30 19:21:08.658465 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.658444    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh"]
Oct 30 19:21:08.689073 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.689030    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 19:21:08.689073 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.689057    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Oct 30 19:21:08.707432 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.707403    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a4ed12fa-0026-4bf8-a99c-83b04caac774-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-8k92x\" (UID: \"a4ed12fa-0026-4bf8-a99c-83b04caac774\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8k92x"
Oct 30 19:21:08.707541 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.707449    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rd95d\" (UniqueName: \"kubernetes.io/projected/a4ed12fa-0026-4bf8-a99c-83b04caac774-kube-api-access-rd95d\") pod \"openshift-state-metrics-7bd7489957-8k92x\" (UID: \"a4ed12fa-0026-4bf8-a99c-83b04caac774\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8k92x"
Oct 30 19:21:08.707541 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.707516    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ffad3bc-f14e-4132-b5ee-3ec84d009252-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-mlpqh\" (UID: \"1ffad3bc-f14e-4132-b5ee-3ec84d009252\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh"
Oct 30 19:21:08.707636 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.707559    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ffad3bc-f14e-4132-b5ee-3ec84d009252-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-mlpqh\" (UID: \"1ffad3bc-f14e-4132-b5ee-3ec84d009252\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh"
Oct 30 19:21:08.707636 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.707588    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4ed12fa-0026-4bf8-a99c-83b04caac774-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-8k92x\" (UID: \"a4ed12fa-0026-4bf8-a99c-83b04caac774\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8k92x"
Oct 30 19:21:08.707706 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.707632    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ffad3bc-f14e-4132-b5ee-3ec84d009252-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-mlpqh\" (UID: \"1ffad3bc-f14e-4132-b5ee-3ec84d009252\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh"
Oct 30 19:21:08.707706 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.707682    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1ffad3bc-f14e-4132-b5ee-3ec84d009252-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-mlpqh\" (UID: \"1ffad3bc-f14e-4132-b5ee-3ec84d009252\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh"
Oct 30 19:21:08.707770 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.707708    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1ffad3bc-f14e-4132-b5ee-3ec84d009252-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-mlpqh\" (UID: \"1ffad3bc-f14e-4132-b5ee-3ec84d009252\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh"
Oct 30 19:21:08.707770 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.707734    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdsrm\" (UniqueName: \"kubernetes.io/projected/1ffad3bc-f14e-4132-b5ee-3ec84d009252-kube-api-access-mdsrm\") pod \"kube-state-metrics-5dc8d7d89b-mlpqh\" (UID: \"1ffad3bc-f14e-4132-b5ee-3ec84d009252\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh"
Oct 30 19:21:08.707770 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.707761    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a4ed12fa-0026-4bf8-a99c-83b04caac774-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-8k92x\" (UID: \"a4ed12fa-0026-4bf8-a99c-83b04caac774\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8k92x"
Oct 30 19:21:08.810408 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.809143    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f1622f2e-319b-4259-b314-a4178b7875f0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:08.810408 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.809195    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a4ed12fa-0026-4bf8-a99c-83b04caac774-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-8k92x\" (UID: \"a4ed12fa-0026-4bf8-a99c-83b04caac774\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8k92x"
Oct 30 19:21:08.810408 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.809225    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1ffad3bc-f14e-4132-b5ee-3ec84d009252-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-mlpqh\" (UID: \"1ffad3bc-f14e-4132-b5ee-3ec84d009252\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh"
Oct 30 19:21:08.810408 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.809260    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4ed12fa-0026-4bf8-a99c-83b04caac774-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-8k92x\" (UID: \"a4ed12fa-0026-4bf8-a99c-83b04caac774\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8k92x"
Oct 30 19:21:08.810408 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.809322    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ffad3bc-f14e-4132-b5ee-3ec84d009252-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-mlpqh\" (UID: \"1ffad3bc-f14e-4132-b5ee-3ec84d009252\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh"
Oct 30 19:21:08.810408 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.809346    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ffad3bc-f14e-4132-b5ee-3ec84d009252-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-mlpqh\" (UID: \"1ffad3bc-f14e-4132-b5ee-3ec84d009252\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh"
Oct 30 19:21:08.810408 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.809370    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ffad3bc-f14e-4132-b5ee-3ec84d009252-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-mlpqh\" (UID: \"1ffad3bc-f14e-4132-b5ee-3ec84d009252\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh"
Oct 30 19:21:08.810408 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.809403    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1622f2e-319b-4259-b314-a4178b7875f0-metrics-client-ca\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:08.810408 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.809429    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f1622f2e-319b-4259-b314-a4178b7875f0-node-exporter-tls\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:08.810408 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.809472    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f1622f2e-319b-4259-b314-a4178b7875f0-node-exporter-wtmp\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:08.810408 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.809497    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmftk\" (UniqueName: \"kubernetes.io/projected/f1622f2e-319b-4259-b314-a4178b7875f0-kube-api-access-wmftk\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:08.810408 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.809530    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdsrm\" (UniqueName: \"kubernetes.io/projected/1ffad3bc-f14e-4132-b5ee-3ec84d009252-kube-api-access-mdsrm\") pod \"kube-state-metrics-5dc8d7d89b-mlpqh\" (UID: \"1ffad3bc-f14e-4132-b5ee-3ec84d009252\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh"
Oct 30 19:21:08.810408 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.809557    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f1622f2e-319b-4259-b314-a4178b7875f0-node-exporter-accelerators-collector-config\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:08.810408 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.809598    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1ffad3bc-f14e-4132-b5ee-3ec84d009252-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-mlpqh\" (UID: \"1ffad3bc-f14e-4132-b5ee-3ec84d009252\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh"
Oct 30 19:21:08.810408 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.809626    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f1622f2e-319b-4259-b314-a4178b7875f0-root\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:08.811183 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.809660    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a4ed12fa-0026-4bf8-a99c-83b04caac774-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-8k92x\" (UID: \"a4ed12fa-0026-4bf8-a99c-83b04caac774\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8k92x"
Oct 30 19:21:08.811183 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.809684    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd95d\" (UniqueName: \"kubernetes.io/projected/a4ed12fa-0026-4bf8-a99c-83b04caac774-kube-api-access-rd95d\") pod \"openshift-state-metrics-7bd7489957-8k92x\" (UID: \"a4ed12fa-0026-4bf8-a99c-83b04caac774\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8k92x"
Oct 30 19:21:08.811183 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.809723    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1622f2e-319b-4259-b314-a4178b7875f0-sys\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:08.811183 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.809754    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f1622f2e-319b-4259-b314-a4178b7875f0-node-exporter-textfile\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:08.811183 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.810824    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1ffad3bc-f14e-4132-b5ee-3ec84d009252-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-mlpqh\" (UID: \"1ffad3bc-f14e-4132-b5ee-3ec84d009252\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh"
Oct 30 19:21:08.811426 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.811244    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1ffad3bc-f14e-4132-b5ee-3ec84d009252-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-mlpqh\" (UID: \"1ffad3bc-f14e-4132-b5ee-3ec84d009252\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh"
Oct 30 19:21:08.811426 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.811358    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ffad3bc-f14e-4132-b5ee-3ec84d009252-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-mlpqh\" (UID: \"1ffad3bc-f14e-4132-b5ee-3ec84d009252\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh"
Oct 30 19:21:08.811893 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.811611    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4ed12fa-0026-4bf8-a99c-83b04caac774-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-8k92x\" (UID: \"a4ed12fa-0026-4bf8-a99c-83b04caac774\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8k92x"
Oct 30 19:21:08.812937 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.812912    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a4ed12fa-0026-4bf8-a99c-83b04caac774-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-8k92x\" (UID: \"a4ed12fa-0026-4bf8-a99c-83b04caac774\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8k92x"
Oct 30 19:21:08.813878 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.813845    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ffad3bc-f14e-4132-b5ee-3ec84d009252-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-mlpqh\" (UID: \"1ffad3bc-f14e-4132-b5ee-3ec84d009252\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh"
Oct 30 19:21:08.814298 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.814283    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ffad3bc-f14e-4132-b5ee-3ec84d009252-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-mlpqh\" (UID: \"1ffad3bc-f14e-4132-b5ee-3ec84d009252\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh"
Oct 30 19:21:08.816331 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.816306    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a4ed12fa-0026-4bf8-a99c-83b04caac774-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-8k92x\" (UID: \"a4ed12fa-0026-4bf8-a99c-83b04caac774\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8k92x"
Oct 30 19:21:08.834656 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.834629    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdsrm\" (UniqueName: \"kubernetes.io/projected/1ffad3bc-f14e-4132-b5ee-3ec84d009252-kube-api-access-mdsrm\") pod \"kube-state-metrics-5dc8d7d89b-mlpqh\" (UID: \"1ffad3bc-f14e-4132-b5ee-3ec84d009252\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh"
Oct 30 19:21:08.834939 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.834914    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rd95d\" (UniqueName: \"kubernetes.io/projected/a4ed12fa-0026-4bf8-a99c-83b04caac774-kube-api-access-rd95d\") pod \"openshift-state-metrics-7bd7489957-8k92x\" (UID: \"a4ed12fa-0026-4bf8-a99c-83b04caac774\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8k92x"
Oct 30 19:21:08.888892 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.886906    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8k92x"
Oct 30 19:21:08.910747 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.910713    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1622f2e-319b-4259-b314-a4178b7875f0-sys\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:08.910893 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.910766    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f1622f2e-319b-4259-b314-a4178b7875f0-node-exporter-textfile\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:08.910893 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.910802    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f1622f2e-319b-4259-b314-a4178b7875f0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:08.910967 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.910912    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1622f2e-319b-4259-b314-a4178b7875f0-metrics-client-ca\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:08.910967 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.910940    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f1622f2e-319b-4259-b314-a4178b7875f0-node-exporter-tls\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:08.911076 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.910977    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f1622f2e-319b-4259-b314-a4178b7875f0-node-exporter-wtmp\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:08.911076 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.911002    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmftk\" (UniqueName: \"kubernetes.io/projected/f1622f2e-319b-4259-b314-a4178b7875f0-kube-api-access-wmftk\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:08.911076 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.911028    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f1622f2e-319b-4259-b314-a4178b7875f0-node-exporter-accelerators-collector-config\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:08.911076 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.911059    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f1622f2e-319b-4259-b314-a4178b7875f0-root\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:08.911223 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.911135    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f1622f2e-319b-4259-b314-a4178b7875f0-root\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:08.911223 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.911186    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1622f2e-319b-4259-b314-a4178b7875f0-sys\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:08.911522 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.911502    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f1622f2e-319b-4259-b314-a4178b7875f0-node-exporter-textfile\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:08.912037 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.911999    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f1622f2e-319b-4259-b314-a4178b7875f0-node-exporter-wtmp\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:08.912512 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.912490    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1622f2e-319b-4259-b314-a4178b7875f0-metrics-client-ca\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:08.912933 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.912908    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f1622f2e-319b-4259-b314-a4178b7875f0-node-exporter-accelerators-collector-config\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:08.914565 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.914541    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f1622f2e-319b-4259-b314-a4178b7875f0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:08.914965 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.914940    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f1622f2e-319b-4259-b314-a4178b7875f0-node-exporter-tls\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:08.933732 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.932918    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmftk\" (UniqueName: \"kubernetes.io/projected/f1622f2e-319b-4259-b314-a4178b7875f0-kube-api-access-wmftk\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:08.934311 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.934115    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh"
Oct 30 19:21:08.941831 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:08.941790    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:09.215409 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:09.215379    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-8k92x"]
Oct 30 19:21:09.221092 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:21:09.221057    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4ed12fa_0026_4bf8_a99c_83b04caac774.slice/crio-02f8343e26b330af5d55a4da95611a92cafd596b71d3f7a2b502f1d64f3f2e6d WatchSource:0}: Error finding container 02f8343e26b330af5d55a4da95611a92cafd596b71d3f7a2b502f1d64f3f2e6d: Status 404 returned error can't find the container with id 02f8343e26b330af5d55a4da95611a92cafd596b71d3f7a2b502f1d64f3f2e6d
Oct 30 19:21:09.250016 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:09.249975    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p8nq5" event={"ID":"9a1d2cec-f539-4106-a185-3134a3df747c","Type":"ContainerStarted","Data":"996c4cd5fdea554213d940511d37ebe93ddfc30a51e3d6063c1d42807a6b12a9"}
Oct 30 19:21:09.252301 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:09.252272    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vn5fp" event={"ID":"4b70ef66-ed5b-4917-8bca-36757fee37e3","Type":"ContainerStarted","Data":"ec94b6e72bdbbdb97147124723895b5fe167a8257505a3c493e199586f416f31"}
Oct 30 19:21:09.252413 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:09.252326    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vn5fp" event={"ID":"4b70ef66-ed5b-4917-8bca-36757fee37e3","Type":"ContainerStarted","Data":"0459ee2216a238dc680c811ab116ad512fb10dd50f63e18aa414243005305ac3"}
Oct 30 19:21:09.252470 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:09.252448    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vn5fp"
Oct 30 19:21:09.254574 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:09.254544    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jqs6k" event={"ID":"f1622f2e-319b-4259-b314-a4178b7875f0","Type":"ContainerStarted","Data":"ebe32d9b522ba42b347de33cfc25c86867ad907ece79d0455ede7a3820a1c32d"}
Oct 30 19:21:09.256080 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:09.256019    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8k92x" event={"ID":"a4ed12fa-0026-4bf8-a99c-83b04caac774","Type":"ContainerStarted","Data":"02f8343e26b330af5d55a4da95611a92cafd596b71d3f7a2b502f1d64f3f2e6d"}
Oct 30 19:21:09.261283 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:09.261257    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh"]
Oct 30 19:21:09.289791 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:09.289661    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-p8nq5" podStartSLOduration=1.8716481470000002 podStartE2EDuration="4.289640775s" totalImagesPullingTime="2.417992628s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:21:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:09.288570685 +0000 UTC m=+52.844792961" watchObservedRunningTime="2026-10-30 19:21:09.289640775 +0000 UTC m=+52.845863055"
Oct 30 19:21:09.342510 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:09.342451    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-vn5fp" podStartSLOduration=2.192090681 podStartE2EDuration="4.342433976s" totalImagesPullingTime="2.150343295s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:09.341958079 +0000 UTC m=+52.898180356" watchObservedRunningTime="2026-10-30 19:21:09.342433976 +0000 UTC m=+52.898656254"
Oct 30 19:21:09.989596 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:09.987949    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 19:21:09.993718 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:09.991989    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.025901 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.025051    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 19:21:10.025901 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.025306    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 19:21:10.025901 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.025343    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 19:21:10.025901 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.025386    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 19:21:10.025901 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.025593    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 19:21:10.025901 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.025670    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 19:21:10.025901 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.025785    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-qh4dk\""
Oct 30 19:21:10.025901 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.025821    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 19:21:10.025901 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.025845    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 19:21:10.026633 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.026568    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 19:21:10.070587 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.066991    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 19:21:10.122266 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.122229    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-config-volume\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.122266 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.122276    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.122590 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.122437    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/40121914-4632-47b7-bc42-8299b3d6d6bd-config-out\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.122590 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.122474    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-web-config\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.122590 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.122506    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.122590 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.122556    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.122793 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.122668    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vlf5\" (UniqueName: \"kubernetes.io/projected/40121914-4632-47b7-bc42-8299b3d6d6bd-kube-api-access-6vlf5\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.122793 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.122720    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/40121914-4632-47b7-bc42-8299b3d6d6bd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.122793 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.122751    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.122793 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.122776    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40121914-4632-47b7-bc42-8299b3d6d6bd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.123076 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.122804    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.123076 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.122849    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/40121914-4632-47b7-bc42-8299b3d6d6bd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.123076 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.122905    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40121914-4632-47b7-bc42-8299b3d6d6bd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.224461 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.224255    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.224461 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.224299    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40121914-4632-47b7-bc42-8299b3d6d6bd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.224461 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.224330    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.224461 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.224378    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/40121914-4632-47b7-bc42-8299b3d6d6bd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.224461 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.224414    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40121914-4632-47b7-bc42-8299b3d6d6bd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.224461 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.224448    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-config-volume\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.224802 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.224472    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.224802 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.224541    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/40121914-4632-47b7-bc42-8299b3d6d6bd-config-out\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.224802 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.224564    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-web-config\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.224802 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.224589    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.224802 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.224616    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.224802 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.224653    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vlf5\" (UniqueName: \"kubernetes.io/projected/40121914-4632-47b7-bc42-8299b3d6d6bd-kube-api-access-6vlf5\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.224802 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.224695    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/40121914-4632-47b7-bc42-8299b3d6d6bd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.225876 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.225835    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/40121914-4632-47b7-bc42-8299b3d6d6bd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.230080 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.226724    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40121914-4632-47b7-bc42-8299b3d6d6bd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.230080 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.227517    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40121914-4632-47b7-bc42-8299b3d6d6bd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.232353 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.232299    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.232892 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.232850    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-config-volume\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.233418 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.233350    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.233418 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.233373    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/40121914-4632-47b7-bc42-8299b3d6d6bd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.234184 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.234142    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-web-config\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.234670 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.234652    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.234850 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.234794    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/40121914-4632-47b7-bc42-8299b3d6d6bd-config-out\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.235251 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.235169    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.237318 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.237228    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.252724 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.252647    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vlf5\" (UniqueName: \"kubernetes.io/projected/40121914-4632-47b7-bc42-8299b3d6d6bd-kube-api-access-6vlf5\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.270602 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.269896    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jqs6k" event={"ID":"f1622f2e-319b-4259-b314-a4178b7875f0","Type":"ContainerStarted","Data":"29fa2072d656e4a258caef40f204d7257e886f5786ef6a225be8179331270a14"}
Oct 30 19:21:10.271815 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.271792    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8k92x" event={"ID":"a4ed12fa-0026-4bf8-a99c-83b04caac774","Type":"ContainerStarted","Data":"8351972bc605b895746903c7aa7a27e7300e4a9f4b53cb496651f935a8311250"}
Oct 30 19:21:10.271980 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.271932    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8k92x" event={"ID":"a4ed12fa-0026-4bf8-a99c-83b04caac774","Type":"ContainerStarted","Data":"2c265612c28d85424d5d7100b9e9b1b1342a55d02b8bc4d56aa1a3f37e4772f1"}
Oct 30 19:21:10.274256 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.274192    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh" event={"ID":"1ffad3bc-f14e-4132-b5ee-3ec84d009252","Type":"ContainerStarted","Data":"f9f6d62c512fcf71a180244d70043254b9c542a889ca7b3feecd9db2ef1a70d2"}
Oct 30 19:21:10.287082 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.287055    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-q2cc6" event={"ID":"c59e8b44-50b7-4a50-af6a-1bb9a7547a56","Type":"ContainerStarted","Data":"836034ae0835cf63aa2c57d4f1c90526f38087b9c39505fea489d0aa45a503ab"}
Oct 30 19:21:10.315213 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.315180    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:10.346762 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.346707    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-q2cc6" podStartSLOduration=2.328610948 podStartE2EDuration="5.346688767s" totalImagesPullingTime="3.018077819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:10.345549325 +0000 UTC m=+53.901771680" watchObservedRunningTime="2026-10-30 19:21:10.346688767 +0000 UTC m=+53.902911043"
Oct 30 19:21:10.646606 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.646503    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 19:21:10.650543 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:21:10.650511    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40121914_4632_47b7_bc42_8299b3d6d6bd.slice/crio-990f36d93789ddcfa3d80eae721de26344c8e54c43783b60e92e3128c3a8d2ae WatchSource:0}: Error finding container 990f36d93789ddcfa3d80eae721de26344c8e54c43783b60e92e3128c3a8d2ae: Status 404 returned error can't find the container with id 990f36d93789ddcfa3d80eae721de26344c8e54c43783b60e92e3128c3a8d2ae
Oct 30 19:21:10.775195 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.775162    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-66d96d76c5-nmsds"]
Oct 30 19:21:10.778566 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.778538    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:10.795033 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.795007    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Oct 30 19:21:10.795033 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.795030    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-4roo0a8peihja\""
Oct 30 19:21:10.795033 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.795007    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Oct 30 19:21:10.795276 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.795009    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Oct 30 19:21:10.795276 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.795102    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Oct 30 19:21:10.795276 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.795111    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Oct 30 19:21:10.795276 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.795126    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-xfcg7\""
Oct 30 19:21:10.805682 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.805659    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-66d96d76c5-nmsds"]
Oct 30 19:21:10.930661 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.930628    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:10.930810 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.930689    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:10.930810 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.930725    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:10.930810 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.930752    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqrkv\" (UniqueName: \"kubernetes.io/projected/3b05813e-ce3d-4e71-a64e-7417997c2988-kube-api-access-jqrkv\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:10.930810 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.930788    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-grpc-tls\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:10.931052 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.930813    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b05813e-ce3d-4e71-a64e-7417997c2988-tmp\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:10.931052 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.930878    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:10.931052 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.930914    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-thanos-querier-tls\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:10.931052 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:10.930937    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3b05813e-ce3d-4e71-a64e-7417997c2988-metrics-client-ca\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:11.032937 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:11.032905    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:11.033072 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:11.032944    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqrkv\" (UniqueName: \"kubernetes.io/projected/3b05813e-ce3d-4e71-a64e-7417997c2988-kube-api-access-jqrkv\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:11.033072 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:11.032976    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-grpc-tls\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:11.033072 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:11.033001    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b05813e-ce3d-4e71-a64e-7417997c2988-tmp\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:11.033072 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:11.033047    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:11.033279 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:11.033080    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-thanos-querier-tls\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:11.033279 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:11.033103    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3b05813e-ce3d-4e71-a64e-7417997c2988-metrics-client-ca\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:11.033279 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:11.033185    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:11.033279 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:11.033229    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:11.034492 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:11.034345    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3b05813e-ce3d-4e71-a64e-7417997c2988-metrics-client-ca\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:11.034492 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:11.034373    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b05813e-ce3d-4e71-a64e-7417997c2988-tmp\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:11.036283 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:11.036259    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:11.036665 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:11.036637    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-thanos-querier-tls\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:11.038075 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:11.038039    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:11.038418 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:11.038393    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:11.038501 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:11.038432    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:11.039947 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:11.039902    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-grpc-tls\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:11.066849 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:11.066818    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqrkv\" (UniqueName: \"kubernetes.io/projected/3b05813e-ce3d-4e71-a64e-7417997c2988-kube-api-access-jqrkv\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:11.104081 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:11.104049    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:11.292489 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:11.292457    2574 generic.go:356] "Generic (PLEG): container finished" podID="f1622f2e-319b-4259-b314-a4178b7875f0" containerID="29fa2072d656e4a258caef40f204d7257e886f5786ef6a225be8179331270a14" exitCode=0
Oct 30 19:21:11.292920 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:11.292746    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jqs6k" event={"ID":"f1622f2e-319b-4259-b314-a4178b7875f0","Type":"ContainerDied","Data":"29fa2072d656e4a258caef40f204d7257e886f5786ef6a225be8179331270a14"}
Oct 30 19:21:11.296038 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:11.295965    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8k92x" event={"ID":"a4ed12fa-0026-4bf8-a99c-83b04caac774","Type":"ContainerStarted","Data":"2f3e45a68c23fbb7172297a695045ce338d38f751f8abf6cf7826837c58bfe18"}
Oct 30 19:21:11.299208 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:11.299181    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh" event={"ID":"1ffad3bc-f14e-4132-b5ee-3ec84d009252","Type":"ContainerStarted","Data":"00254d2687bef7a0fa2ea569c26b10a23d328a195a1b57f58a7490288241373a"}
Oct 30 19:21:11.299314 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:11.299235    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh" event={"ID":"1ffad3bc-f14e-4132-b5ee-3ec84d009252","Type":"ContainerStarted","Data":"7afc64592941d2eeb18e12d847d7797ebadc543ab9aa23bdfb1289d37963014f"}
Oct 30 19:21:11.299314 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:11.299262    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh" event={"ID":"1ffad3bc-f14e-4132-b5ee-3ec84d009252","Type":"ContainerStarted","Data":"f48ddbd6f4b6223199144c23e5e978aafb5f4d3ddde7b0d24e8144c2a10e09b3"}
Oct 30 19:21:11.300965 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:11.300840    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"40121914-4632-47b7-bc42-8299b3d6d6bd","Type":"ContainerStarted","Data":"990f36d93789ddcfa3d80eae721de26344c8e54c43783b60e92e3128c3a8d2ae"}
Oct 30 19:21:11.376331 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:11.375424    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh" podStartSLOduration=1.8291822500000001 podStartE2EDuration="3.375406043s" totalImagesPullingTime="1.546223793s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:11.373768123 +0000 UTC m=+54.929990421" watchObservedRunningTime="2026-10-30 19:21:11.375406043 +0000 UTC m=+54.931628321"
Oct 30 19:21:11.414071 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:11.413957    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8k92x" podStartSLOduration=2.140289692 podStartE2EDuration="3.413940246s" totalImagesPullingTime="1.273650554s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:11.413121968 +0000 UTC m=+54.969344264" watchObservedRunningTime="2026-10-30 19:21:11.413940246 +0000 UTC m=+54.970162523"
Oct 30 19:21:11.469741 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:11.469612    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-66d96d76c5-nmsds"]
Oct 30 19:21:11.475910 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:21:11.475851    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b05813e_ce3d_4e71_a64e_7417997c2988.slice/crio-ab80f0926c3314624b09b3a9706dc0fcd4de7fd2272425a7b5e1cc8744e85053 WatchSource:0}: Error finding container ab80f0926c3314624b09b3a9706dc0fcd4de7fd2272425a7b5e1cc8744e85053: Status 404 returned error can't find the container with id ab80f0926c3314624b09b3a9706dc0fcd4de7fd2272425a7b5e1cc8744e85053
Oct 30 19:21:12.306236 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:12.306193    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d79db5f98-fmng5" event={"ID":"44b9fc01-6837-46cd-bf01-40cb8fcb90f5","Type":"ContainerStarted","Data":"213d8582e8b2772f3ea9d89844f4f2694c46a2701148a17cd72a3317666313c7"}
Oct 30 19:21:12.309355 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:12.308717    2574 generic.go:356] "Generic (PLEG): container finished" podID="40121914-4632-47b7-bc42-8299b3d6d6bd" containerID="65ce9926fc3f76d0f605f9b84cd747beac60ac1ea3c9e9b2a10d37644104a416" exitCode=0
Oct 30 19:21:12.309355 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:12.308812    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"40121914-4632-47b7-bc42-8299b3d6d6bd","Type":"ContainerDied","Data":"65ce9926fc3f76d0f605f9b84cd747beac60ac1ea3c9e9b2a10d37644104a416"}
Oct 30 19:21:12.313008 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:12.312983    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jqs6k" event={"ID":"f1622f2e-319b-4259-b314-a4178b7875f0","Type":"ContainerStarted","Data":"ce1c0ef0dbd6a76c57ead8f8a7c26212a41b65ccb3627da1b358bcfa79fd7cf4"}
Oct 30 19:21:12.313089 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:12.313033    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jqs6k" event={"ID":"f1622f2e-319b-4259-b314-a4178b7875f0","Type":"ContainerStarted","Data":"ffd3f93aaa2ce466736271165189dfb1ace85bbaecab157c53b4be2b716255cd"}
Oct 30 19:21:12.314697 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:12.314671    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds" event={"ID":"3b05813e-ce3d-4e71-a64e-7417997c2988","Type":"ContainerStarted","Data":"ab80f0926c3314624b09b3a9706dc0fcd4de7fd2272425a7b5e1cc8744e85053"}
Oct 30 19:21:12.392933 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:12.392883    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-5d79db5f98-fmng5" podStartSLOduration=1.723988276 podStartE2EDuration="5.392848879s" totalImagesPullingTime="3.668860603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:12.344603991 +0000 UTC m=+55.900826268" watchObservedRunningTime="2026-10-30 19:21:12.392848879 +0000 UTC m=+55.949071156"
Oct 30 19:21:12.432277 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:12.432219    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-jqs6k" podStartSLOduration=3.157029633 podStartE2EDuration="4.432200313s" totalImagesPullingTime="1.171095463s" totalInitContainerRuntime="104.075217ms" isStatefulPod=true podCreationTimestamp="2026-10-30 19:21:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:12.432099972 +0000 UTC m=+55.988322249" watchObservedRunningTime="2026-10-30 19:21:12.432200313 +0000 UTC m=+55.988422595"
Oct 30 19:21:13.070216 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.070179    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6b87cd958c-prt2j"]
Oct 30 19:21:13.074120 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.074099    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:13.088003 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.087980    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Oct 30 19:21:13.088003 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.087995    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Oct 30 19:21:13.088190 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.088091    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 19:21:13.098765 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.098741    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-xvh7n\""
Oct 30 19:21:13.098911 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.098794    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-3hcakgtj1qoqn\""
Oct 30 19:21:13.098911 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.098845    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Oct 30 19:21:13.103053 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.103031    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6b87cd958c-prt2j"]
Oct 30 19:21:13.157540 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.157497    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slnw6\" (UniqueName: \"kubernetes.io/projected/2ca12196-1929-4ece-a076-d36527b95ff1-kube-api-access-slnw6\") pod \"metrics-server-6b87cd958c-prt2j\" (UID: \"2ca12196-1929-4ece-a076-d36527b95ff1\") " pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:13.157719 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.157576    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ca12196-1929-4ece-a076-d36527b95ff1-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6b87cd958c-prt2j\" (UID: \"2ca12196-1929-4ece-a076-d36527b95ff1\") " pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:13.157719 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.157609    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2ca12196-1929-4ece-a076-d36527b95ff1-client-ca-bundle\") pod \"metrics-server-6b87cd958c-prt2j\" (UID: \"2ca12196-1929-4ece-a076-d36527b95ff1\") " pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:13.157839 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.157718    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2ca12196-1929-4ece-a076-d36527b95ff1-audit-log\") pod \"metrics-server-6b87cd958c-prt2j\" (UID: \"2ca12196-1929-4ece-a076-d36527b95ff1\") " pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:13.157839 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.157788    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2ca12196-1929-4ece-a076-d36527b95ff1-metrics-server-audit-profiles\") pod \"metrics-server-6b87cd958c-prt2j\" (UID: \"2ca12196-1929-4ece-a076-d36527b95ff1\") " pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:13.157839 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.157822    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2ca12196-1929-4ece-a076-d36527b95ff1-secret-metrics-server-client-certs\") pod \"metrics-server-6b87cd958c-prt2j\" (UID: \"2ca12196-1929-4ece-a076-d36527b95ff1\") " pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:13.158034 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.157897    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2ca12196-1929-4ece-a076-d36527b95ff1-secret-metrics-server-tls\") pod \"metrics-server-6b87cd958c-prt2j\" (UID: \"2ca12196-1929-4ece-a076-d36527b95ff1\") " pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:13.259246 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.259208    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2ca12196-1929-4ece-a076-d36527b95ff1-audit-log\") pod \"metrics-server-6b87cd958c-prt2j\" (UID: \"2ca12196-1929-4ece-a076-d36527b95ff1\") " pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:13.259415 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.259271    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2ca12196-1929-4ece-a076-d36527b95ff1-metrics-server-audit-profiles\") pod \"metrics-server-6b87cd958c-prt2j\" (UID: \"2ca12196-1929-4ece-a076-d36527b95ff1\") " pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:13.259415 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.259304    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2ca12196-1929-4ece-a076-d36527b95ff1-secret-metrics-server-client-certs\") pod \"metrics-server-6b87cd958c-prt2j\" (UID: \"2ca12196-1929-4ece-a076-d36527b95ff1\") " pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:13.259415 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.259332    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2ca12196-1929-4ece-a076-d36527b95ff1-secret-metrics-server-tls\") pod \"metrics-server-6b87cd958c-prt2j\" (UID: \"2ca12196-1929-4ece-a076-d36527b95ff1\") " pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:13.259415 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.259395    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slnw6\" (UniqueName: \"kubernetes.io/projected/2ca12196-1929-4ece-a076-d36527b95ff1-kube-api-access-slnw6\") pod \"metrics-server-6b87cd958c-prt2j\" (UID: \"2ca12196-1929-4ece-a076-d36527b95ff1\") " pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:13.259544 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.259431    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ca12196-1929-4ece-a076-d36527b95ff1-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6b87cd958c-prt2j\" (UID: \"2ca12196-1929-4ece-a076-d36527b95ff1\") " pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:13.259544 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.259446    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2ca12196-1929-4ece-a076-d36527b95ff1-client-ca-bundle\") pod \"metrics-server-6b87cd958c-prt2j\" (UID: \"2ca12196-1929-4ece-a076-d36527b95ff1\") " pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:13.259884 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.259803    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2ca12196-1929-4ece-a076-d36527b95ff1-audit-log\") pod \"metrics-server-6b87cd958c-prt2j\" (UID: \"2ca12196-1929-4ece-a076-d36527b95ff1\") " pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:13.260317 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.260294    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ca12196-1929-4ece-a076-d36527b95ff1-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6b87cd958c-prt2j\" (UID: \"2ca12196-1929-4ece-a076-d36527b95ff1\") " pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:13.260839 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.260783    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2ca12196-1929-4ece-a076-d36527b95ff1-metrics-server-audit-profiles\") pod \"metrics-server-6b87cd958c-prt2j\" (UID: \"2ca12196-1929-4ece-a076-d36527b95ff1\") " pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:13.262410 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.262390    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2ca12196-1929-4ece-a076-d36527b95ff1-secret-metrics-server-client-certs\") pod \"metrics-server-6b87cd958c-prt2j\" (UID: \"2ca12196-1929-4ece-a076-d36527b95ff1\") " pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:13.262510 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.262449    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2ca12196-1929-4ece-a076-d36527b95ff1-secret-metrics-server-tls\") pod \"metrics-server-6b87cd958c-prt2j\" (UID: \"2ca12196-1929-4ece-a076-d36527b95ff1\") " pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:13.263558 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.263534    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2ca12196-1929-4ece-a076-d36527b95ff1-client-ca-bundle\") pod \"metrics-server-6b87cd958c-prt2j\" (UID: \"2ca12196-1929-4ece-a076-d36527b95ff1\") " pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:13.278815 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.278788    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-slnw6\" (UniqueName: \"kubernetes.io/projected/2ca12196-1929-4ece-a076-d36527b95ff1-kube-api-access-slnw6\") pod \"metrics-server-6b87cd958c-prt2j\" (UID: \"2ca12196-1929-4ece-a076-d36527b95ff1\") " pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:13.347386 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.345919    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-mffps"]
Oct 30 19:21:13.349656 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.349632    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-mffps"
Oct 30 19:21:13.362829 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.360945    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/aa367dd7-da29-424e-9acc-7706f4377494-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-mffps\" (UID: \"aa367dd7-da29-424e-9acc-7706f4377494\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-mffps"
Oct 30 19:21:13.362829 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.361001    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9wwz\" (UniqueName: \"kubernetes.io/projected/aa367dd7-da29-424e-9acc-7706f4377494-kube-api-access-s9wwz\") pod \"monitoring-plugin-77f4f76594-mffps\" (UID: \"aa367dd7-da29-424e-9acc-7706f4377494\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-mffps"
Oct 30 19:21:13.366967 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.366942    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Oct 30 19:21:13.367213 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.367194    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-rv27s\""
Oct 30 19:21:13.374305 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.374282    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-mffps"]
Oct 30 19:21:13.389470 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.389238    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:13.463010 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.462080    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/aa367dd7-da29-424e-9acc-7706f4377494-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-mffps\" (UID: \"aa367dd7-da29-424e-9acc-7706f4377494\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-mffps"
Oct 30 19:21:13.463010 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.462141    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9wwz\" (UniqueName: \"kubernetes.io/projected/aa367dd7-da29-424e-9acc-7706f4377494-kube-api-access-s9wwz\") pod \"monitoring-plugin-77f4f76594-mffps\" (UID: \"aa367dd7-da29-424e-9acc-7706f4377494\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-mffps"
Oct 30 19:21:13.465923 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.465848    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/aa367dd7-da29-424e-9acc-7706f4377494-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-mffps\" (UID: \"aa367dd7-da29-424e-9acc-7706f4377494\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-mffps"
Oct 30 19:21:13.481316 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.481264    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9wwz\" (UniqueName: \"kubernetes.io/projected/aa367dd7-da29-424e-9acc-7706f4377494-kube-api-access-s9wwz\") pod \"monitoring-plugin-77f4f76594-mffps\" (UID: \"aa367dd7-da29-424e-9acc-7706f4377494\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-mffps"
Oct 30 19:21:13.664215 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.664188    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-mffps"
Oct 30 19:21:13.710886 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.709773    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6b87cd958c-prt2j"]
Oct 30 19:21:13.716407 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:21:13.716371    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ca12196_1929_4ece_a076_d36527b95ff1.slice/crio-9653fcfafb20e3486e0d7dff2ce47bd26c384f2b0b50f8f065899a174fc3c0b6 WatchSource:0}: Error finding container 9653fcfafb20e3486e0d7dff2ce47bd26c384f2b0b50f8f065899a174fc3c0b6: Status 404 returned error can't find the container with id 9653fcfafb20e3486e0d7dff2ce47bd26c384f2b0b50f8f065899a174fc3c0b6
Oct 30 19:21:13.967716 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:13.967692    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-mffps"]
Oct 30 19:21:13.975729 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:21:13.975698    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa367dd7_da29_424e_9acc_7706f4377494.slice/crio-454d4e22040463e0ef128cd01598ac2291d8257d485ab2493387e223af92cf71 WatchSource:0}: Error finding container 454d4e22040463e0ef128cd01598ac2291d8257d485ab2493387e223af92cf71: Status 404 returned error can't find the container with id 454d4e22040463e0ef128cd01598ac2291d8257d485ab2493387e223af92cf71
Oct 30 19:21:14.325490 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:14.325450    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j" event={"ID":"2ca12196-1929-4ece-a076-d36527b95ff1","Type":"ContainerStarted","Data":"9653fcfafb20e3486e0d7dff2ce47bd26c384f2b0b50f8f065899a174fc3c0b6"}
Oct 30 19:21:14.330220 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:14.330194    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"40121914-4632-47b7-bc42-8299b3d6d6bd","Type":"ContainerStarted","Data":"e2099a798a3a67494fbe9d71f71a4e97f9e4af74aec44ef1cab6314ef040ec4b"}
Oct 30 19:21:14.330360 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:14.330266    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"40121914-4632-47b7-bc42-8299b3d6d6bd","Type":"ContainerStarted","Data":"c186ea0bc6198666b986912896187eb7190024431a7e9c87b41837b675f2a668"}
Oct 30 19:21:14.330360 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:14.330308    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"40121914-4632-47b7-bc42-8299b3d6d6bd","Type":"ContainerStarted","Data":"49fce01f5692865174f6bd212e20c9f5afd439845a6593959cf5520dbf2fdd5d"}
Oct 30 19:21:14.330360 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:14.330347    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"40121914-4632-47b7-bc42-8299b3d6d6bd","Type":"ContainerStarted","Data":"cbab46292c95c2c7cc8ac5d2214a7463e84a46a9a7fb1fe8698069253353dc88"}
Oct 30 19:21:14.330533 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:14.330383    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"40121914-4632-47b7-bc42-8299b3d6d6bd","Type":"ContainerStarted","Data":"ef67942c95d5a202367f431a4db2814cf5e0648e6fd022bdb67f1d4dca39abd3"}
Oct 30 19:21:14.331920 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:14.331891    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-mffps" event={"ID":"aa367dd7-da29-424e-9acc-7706f4377494","Type":"ContainerStarted","Data":"454d4e22040463e0ef128cd01598ac2291d8257d485ab2493387e223af92cf71"}
Oct 30 19:21:14.335259 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:14.335103    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds" event={"ID":"3b05813e-ce3d-4e71-a64e-7417997c2988","Type":"ContainerStarted","Data":"e4f97b8ffe3c052796ac54e1ab98aa7ef845d90c8f8e7c69e0b12eee19dd5016"}
Oct 30 19:21:14.335259 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:14.335180    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds" event={"ID":"3b05813e-ce3d-4e71-a64e-7417997c2988","Type":"ContainerStarted","Data":"a8ae917672a425ae25283d5cdc9a4e51155c23cd004541ad7768efdb20c7e00c"}
Oct 30 19:21:14.335259 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:14.335219    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds" event={"ID":"3b05813e-ce3d-4e71-a64e-7417997c2988","Type":"ContainerStarted","Data":"f0253f147527a7d4108c340a85cac27b283d2d75f205e2a31210c0a4262f7bd1"}
Oct 30 19:21:14.558738 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:14.558651    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:15.106220 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.106113    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-6bdc9bd9c6-mww4x"]
Oct 30 19:21:15.109778 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.109751    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:15.143003 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.142952    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 30 19:21:15.144793 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.144765    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bdc9bd9c6-mww4x"]
Oct 30 19:21:15.183065 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.183030    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-trusted-ca-bundle\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:15.183252 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.183106    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-console-oauth-config\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:15.183252 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.183156    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-oauth-serving-cert\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:15.183364 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.183253    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-service-ca\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:15.183364 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.183308    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-console-config\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:15.183364 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.183337    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-console-serving-cert\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:15.183483 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.183432    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vv49j\" (UniqueName: \"kubernetes.io/projected/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-kube-api-access-vv49j\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:15.183483 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.183466    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-tmp\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:15.284587 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.284071    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-trusted-ca-bundle\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:15.284587 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.284147    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-console-oauth-config\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:15.284587 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.284288    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-oauth-serving-cert\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:15.285149 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.285123    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-trusted-ca-bundle\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:15.285282 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.285259    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-oauth-serving-cert\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:15.285591 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.285568    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-service-ca\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:15.285682 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.285606    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-console-config\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:15.285682 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.285650    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-console-serving-cert\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:15.285840 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.285700    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv49j\" (UniqueName: \"kubernetes.io/projected/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-kube-api-access-vv49j\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:15.285840 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.285728    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-tmp\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:15.286385 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.286334    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-service-ca\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:15.286488 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.286470    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-tmp\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:15.286781 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.286761    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-console-config\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:15.287211 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.287189    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-console-oauth-config\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:15.288910 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.288852    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-console-serving-cert\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:15.305989 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.305967    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vv49j\" (UniqueName: \"kubernetes.io/projected/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-kube-api-access-vv49j\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:15.342748 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.342707    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds" event={"ID":"3b05813e-ce3d-4e71-a64e-7417997c2988","Type":"ContainerStarted","Data":"a76ce92c8a63a6a6f0c66318cb468a5a9287ee3d8ca36754ee779e3c586d8e74"}
Oct 30 19:21:15.342941 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.342789    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds" event={"ID":"3b05813e-ce3d-4e71-a64e-7417997c2988","Type":"ContainerStarted","Data":"ea18d391a2d4f76d38292eb11c93defdac8774a17c8f5e0a700e220a8966874a"}
Oct 30 19:21:15.342941 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.342827    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds" event={"ID":"3b05813e-ce3d-4e71-a64e-7417997c2988","Type":"ContainerStarted","Data":"b145d6d22fb9bfa5ff6ec15e4a273d93d1d5948e078b03c13aee4d7f0f270ee1"}
Oct 30 19:21:15.342941 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.342926    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:15.344534 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.344505    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j" event={"ID":"2ca12196-1929-4ece-a076-d36527b95ff1","Type":"ContainerStarted","Data":"80c72096221b94d69f9aa289b73b0cf4cc8ee45a0e202dc9e245cd815d593fa3"}
Oct 30 19:21:15.348475 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.348451    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"40121914-4632-47b7-bc42-8299b3d6d6bd","Type":"ContainerStarted","Data":"3be76fada0592d95f1f110abdeb2e3408a0c3e56ef11b553bbba7d7e977eefe1"}
Oct 30 19:21:15.361757 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.361739    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 19:21:15.366198 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.366177    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.380713 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.380690    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 19:21:15.380876 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.380846    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 19:21:15.384855 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.384834    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 19:21:15.385484 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.385444    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds" podStartSLOduration=2.436403768 podStartE2EDuration="5.385426579s" totalImagesPullingTime="2.949022811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:15.385074758 +0000 UTC m=+58.941297075" watchObservedRunningTime="2026-10-30 19:21:15.385426579 +0000 UTC m=+58.941648855"
Oct 30 19:21:15.386335 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.386312    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.386442 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.386349    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.386442 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.386413    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.386545 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.386480    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.386545 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.386520    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.386816 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.386796    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.386915 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.386840    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.386915 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.386904    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-config-out\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.387110 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.386941    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqn8c\" (UniqueName: \"kubernetes.io/projected/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-kube-api-access-pqn8c\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.387110 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.387084    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.387459 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.387441    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.387536 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.387519    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.387817 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.387777    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.387981 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.387959    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.388060 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.388039    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-config\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.388346 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.388135    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.388346 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.388210    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.388346 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.388282    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.388559 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.388540    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.388619 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.388604    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-web-config\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.398538 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.398516    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 19:21:15.424398 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.424359    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j" podStartSLOduration=0.910132213 podStartE2EDuration="2.42434534s" totalImagesPullingTime="1.514213127s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:15.42326822 +0000 UTC m=+58.979490508" watchObservedRunningTime="2026-10-30 19:21:15.42434534 +0000 UTC m=+58.980567617"
Oct 30 19:21:15.424779 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.424762    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:15.483909 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.483021    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 19:21:15.487006 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.485948    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 19:21:15.487006 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.486068    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 19:21:15.487006 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.486075    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 19:21:15.487006 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.486161    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 19:21:15.487006 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.486220    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 19:21:15.487006 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.486264    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 19:21:15.487006 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.486291    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 19:21:15.487006 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.486165    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 19:21:15.487006 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.486476    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 19:21:15.487006 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.486558    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7380srhp40v4j\""
Oct 30 19:21:15.487006 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.486661    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-cqcj8\""
Oct 30 19:21:15.487006 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.486906    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 19:21:15.491688 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.490845    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.491688 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.490896    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.491688 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.490925    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.491688 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.490978    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.491688 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.491004    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-web-config\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.491688 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.491061    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.491688 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.491083    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.491688 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.491109    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.491688 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.491143    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.491688 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.491182    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.491688 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.491224    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.491688 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.491255    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.491688 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.491282    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-config-out\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.491688 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.491305    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqn8c\" (UniqueName: \"kubernetes.io/projected/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-kube-api-access-pqn8c\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.491688 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.491326    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.491688 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.491345    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.491688 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.491363    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.492727 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.491385    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.492727 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.491402    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.492727 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.491419    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-config\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.493232 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.493204    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.493897 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.493578    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.493993 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.493932    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.494803 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.494423    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.494803 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.494460    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.494803 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.494564    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.494803 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.494633    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-config\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.496174 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.495163    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.498235 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.498182    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-web-config\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.498664 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.498636    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.498746 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.498638    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.499350 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.499247    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.500628 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.499696    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.500628 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.499913    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.500628 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.500277    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.500628 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.500593    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-config-out\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.501206 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.501176    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.502539 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.502476    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.503565 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.503535    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.513020 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.512972    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqn8c\" (UniqueName: \"kubernetes.io/projected/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-kube-api-access-pqn8c\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.515957 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.515313    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.233954319 podStartE2EDuration="6.515296877s" totalImagesPullingTime="3.269932751s" totalInitContainerRuntime="11.409807ms" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:15.514092272 +0000 UTC m=+59.070314550" watchObservedRunningTime="2026-10-30 19:21:15.515296877 +0000 UTC m=+59.071519156"
Oct 30 19:21:15.708723 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.708644    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:15.724143 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:15.724120    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bdc9bd9c6-mww4x"]
Oct 30 19:21:15.728499 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:21:15.728468    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55ab4c0b_7882_4bb1_ab3c_75cc7af5ce56.slice/crio-bebff447d4392a1b68f1a0d6050ed163c27381784c1b08ee6f917e60a37309f0 WatchSource:0}: Error finding container bebff447d4392a1b68f1a0d6050ed163c27381784c1b08ee6f917e60a37309f0: Status 404 returned error can't find the container with id bebff447d4392a1b68f1a0d6050ed163c27381784c1b08ee6f917e60a37309f0
Oct 30 19:21:16.030281 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:16.030055    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 19:21:16.034261 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:21:16.034238    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podead8f4e0_54d5_4db6_b7fc_6a5ec203a963.slice/crio-8fe73d8245fc2351fe48fe3f797c40b32b675cb0d0242e738ed1856a41b38627 WatchSource:0}: Error finding container 8fe73d8245fc2351fe48fe3f797c40b32b675cb0d0242e738ed1856a41b38627: Status 404 returned error can't find the container with id 8fe73d8245fc2351fe48fe3f797c40b32b675cb0d0242e738ed1856a41b38627
Oct 30 19:21:16.353610 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:16.353517    2574 generic.go:356] "Generic (PLEG): container finished" podID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerID="595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed" exitCode=0
Oct 30 19:21:16.353750 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:16.353617    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963","Type":"ContainerDied","Data":"595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed"}
Oct 30 19:21:16.353750 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:16.353692    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963","Type":"ContainerStarted","Data":"8fe73d8245fc2351fe48fe3f797c40b32b675cb0d0242e738ed1856a41b38627"}
Oct 30 19:21:16.355306 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:16.355282    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-mffps" event={"ID":"aa367dd7-da29-424e-9acc-7706f4377494","Type":"ContainerStarted","Data":"bd498a7d893c3a4210581132236006e2826741f8479038137560fcf94c003cd7"}
Oct 30 19:21:16.355428 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:16.355417    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-mffps"
Oct 30 19:21:16.357050 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:16.357018    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bdc9bd9c6-mww4x" event={"ID":"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56","Type":"ContainerStarted","Data":"207c492fbd2114ccdc9ec9124790e3531fba903a26c4a742b171c4fd7978e694"}
Oct 30 19:21:16.357165 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:16.357067    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bdc9bd9c6-mww4x" event={"ID":"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56","Type":"ContainerStarted","Data":"bebff447d4392a1b68f1a0d6050ed163c27381784c1b08ee6f917e60a37309f0"}
Oct 30 19:21:16.360405 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:16.360387    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-mffps"
Oct 30 19:21:16.440707 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:16.440656    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-77f4f76594-mffps" podStartSLOduration=2.021775724 podStartE2EDuration="3.440642486s" totalImagesPullingTime="1.418866762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:16.439855313 +0000 UTC m=+59.996077589" watchObservedRunningTime="2026-10-30 19:21:16.440642486 +0000 UTC m=+59.996864763"
Oct 30 19:21:16.477113 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:16.477042    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-6bdc9bd9c6-mww4x" podStartSLOduration=1.47702005 podStartE2EDuration="1.47702005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:16.476492681 +0000 UTC m=+60.032715007" watchObservedRunningTime="2026-10-30 19:21:16.47702005 +0000 UTC m=+60.033242329"
Oct 30 19:21:17.631888 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:17.631823    2574 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:17.632309 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:17.631952    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:17.637743 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:17.637720    2574 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:18.371810 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:18.371784    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:19.291582 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:19.290804    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vn5fp"
Oct 30 19:21:20.381316 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:20.381281    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963","Type":"ContainerStarted","Data":"5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a"}
Oct 30 19:21:20.381728 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:20.381357    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963","Type":"ContainerStarted","Data":"7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16"}
Oct 30 19:21:20.381728 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:20.381386    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963","Type":"ContainerStarted","Data":"59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd"}
Oct 30 19:21:20.381728 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:20.381423    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963","Type":"ContainerStarted","Data":"79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a"}
Oct 30 19:21:20.381728 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:20.381457    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963","Type":"ContainerStarted","Data":"67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2"}
Oct 30 19:21:20.381728 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:20.381492    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963","Type":"ContainerStarted","Data":"a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63"}
Oct 30 19:21:20.447169 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:20.447115    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.280526148 podStartE2EDuration="5.447099926s" totalImagesPullingTime="3.149201446s" totalInitContainerRuntime="17.372332ms" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:20.446274761 +0000 UTC m=+64.002497038" watchObservedRunningTime="2026-10-30 19:21:20.447099926 +0000 UTC m=+64.003322202"
Oct 30 19:21:20.709324 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:20.709222    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:21.364589 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:21.364562    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:22.774182 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:22.774140    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/725d2cce-2a75-426f-8522-d1ddfbabe1b6-original-pull-secret\") pod \"global-pull-secret-syncer-z7mhx\" (UID: \"725d2cce-2a75-426f-8522-d1ddfbabe1b6\") " pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:21:22.774631 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:22.774245    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41ff5ee2-70a2-4baf-af4e-00e6575c2e99-metrics-certs\") pod \"network-metrics-daemon-9jk99\" (UID: \"41ff5ee2-70a2-4baf-af4e-00e6575c2e99\") " pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:21:22.787046 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:22.787009    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 19:21:22.787135 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:22.787057    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 19:21:22.797579 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:22.797546    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/725d2cce-2a75-426f-8522-d1ddfbabe1b6-original-pull-secret\") pod \"global-pull-secret-syncer-z7mhx\" (UID: \"725d2cce-2a75-426f-8522-d1ddfbabe1b6\") " pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:21:22.797677 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:22.797560    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41ff5ee2-70a2-4baf-af4e-00e6575c2e99-metrics-certs\") pod \"network-metrics-daemon-9jk99\" (UID: \"41ff5ee2-70a2-4baf-af4e-00e6575c2e99\") " pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:21:22.875253 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:22.875215    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2xwk\" (UniqueName: \"kubernetes.io/projected/cd444ba1-edf1-49aa-a0cf-b863c93e60be-kube-api-access-c2xwk\") pod \"network-check-target-5j799\" (UID: \"cd444ba1-edf1-49aa-a0cf-b863c93e60be\") " pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:21:22.877661 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:22.877639    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-65nsn\""
Oct 30 19:21:22.885716 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:22.885698    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:21:22.888125 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:22.888103    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 19:21:22.908646 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:22.908624    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 19:21:22.919380 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:22.919354    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2xwk\" (UniqueName: \"kubernetes.io/projected/cd444ba1-edf1-49aa-a0cf-b863c93e60be-kube-api-access-c2xwk\") pod \"network-check-target-5j799\" (UID: \"cd444ba1-edf1-49aa-a0cf-b863c93e60be\") " pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:21:23.072981 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:23.072898    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-77hs9\""
Oct 30 19:21:23.077602 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:23.077582    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sl59z\""
Oct 30 19:21:23.080665 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:23.080651    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:21:23.085616 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:23.085602    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:21:23.186849 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:23.186814    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-z7mhx"]
Oct 30 19:21:23.191231 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:21:23.191203    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod725d2cce_2a75_426f_8522_d1ddfbabe1b6.slice/crio-e6f51ee7798f1661bfe55e1c8b7ce3927025c79b69d52572c9c29603d43dcf53 WatchSource:0}: Error finding container e6f51ee7798f1661bfe55e1c8b7ce3927025c79b69d52572c9c29603d43dcf53: Status 404 returned error can't find the container with id e6f51ee7798f1661bfe55e1c8b7ce3927025c79b69d52572c9c29603d43dcf53
Oct 30 19:21:23.377977 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:23.377933    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5j799"]
Oct 30 19:21:23.382195 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:21:23.382164    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd444ba1_edf1_49aa_a0cf_b863c93e60be.slice/crio-60ebcfd6f800063ca62814f9d5037ff17ab8021bb25a20a9fb23019628c36457 WatchSource:0}: Error finding container 60ebcfd6f800063ca62814f9d5037ff17ab8021bb25a20a9fb23019628c36457: Status 404 returned error can't find the container with id 60ebcfd6f800063ca62814f9d5037ff17ab8021bb25a20a9fb23019628c36457
Oct 30 19:21:23.393827 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:23.393786    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9jk99"]
Oct 30 19:21:23.399779 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:23.399753    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5j799" event={"ID":"cd444ba1-edf1-49aa-a0cf-b863c93e60be","Type":"ContainerStarted","Data":"60ebcfd6f800063ca62814f9d5037ff17ab8021bb25a20a9fb23019628c36457"}
Oct 30 19:21:23.401136 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:23.401104    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-z7mhx" event={"ID":"725d2cce-2a75-426f-8522-d1ddfbabe1b6","Type":"ContainerStarted","Data":"e6f51ee7798f1661bfe55e1c8b7ce3927025c79b69d52572c9c29603d43dcf53"}
Oct 30 19:21:23.401638 ip-10-0-132-152 kubenswrapper[2574]: W1030 19:21:23.401615    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41ff5ee2_70a2_4baf_af4e_00e6575c2e99.slice/crio-90cf8b7e2c8a82be30790ee5f04c3379be42be441410a7ed654100bd85592c48 WatchSource:0}: Error finding container 90cf8b7e2c8a82be30790ee5f04c3379be42be441410a7ed654100bd85592c48: Status 404 returned error can't find the container with id 90cf8b7e2c8a82be30790ee5f04c3379be42be441410a7ed654100bd85592c48
Oct 30 19:21:24.408105 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:24.408063    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9jk99" event={"ID":"41ff5ee2-70a2-4baf-af4e-00e6575c2e99","Type":"ContainerStarted","Data":"90cf8b7e2c8a82be30790ee5f04c3379be42be441410a7ed654100bd85592c48"}
Oct 30 19:21:25.415704 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:25.415666    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9jk99" event={"ID":"41ff5ee2-70a2-4baf-af4e-00e6575c2e99","Type":"ContainerStarted","Data":"aea1f90b0eb2cc739ead201582f60c258482e89942b98c3b1681c5ae19ca1bca"}
Oct 30 19:21:25.416154 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:25.415737    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9jk99" event={"ID":"41ff5ee2-70a2-4baf-af4e-00e6575c2e99","Type":"ContainerStarted","Data":"6e5ee4e1787b9616738e3d4716d32fbb87f55bb23274cb399b3937960c1e74af"}
Oct 30 19:21:25.426198 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:25.425518    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:25.426198 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:25.425583    2574 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:25.432314 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:25.432293    2574 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:25.462094 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:25.462028    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9jk99" podStartSLOduration=67.489624136 podStartE2EDuration="1m8.462008158s" totalImagesPullingTime="972.384022ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:20:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:25.46013986 +0000 UTC m=+69.016362167" watchObservedRunningTime="2026-10-30 19:21:25.462008158 +0000 UTC m=+69.018230437"
Oct 30 19:21:26.427536 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:26.427497    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:26.499996 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:26.499896    2574 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d79db5f98-fmng5"]
Oct 30 19:21:27.427038 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:27.426985    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5j799" event={"ID":"cd444ba1-edf1-49aa-a0cf-b863c93e60be","Type":"ContainerStarted","Data":"dddbeae8218a520bb62107ffc324f3ad3f26ea239c3cf39e58f517c36e53c0fd"}
Oct 30 19:21:27.464928 ip-10-0-132-152 kubenswrapper[2574]: I1030 19:21:27.464843    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5j799" podStartSLOduration=67.282199554 podStartE2EDuration="1m10.464826018s" totalImagesPullingTime="3.182626464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:20:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:27.463251637 +0000 UTC m=+71.019473915" watchObservedRunningTime="2026-10-30 19:21:27.464826018 +0000 UTC m=+71.021048295"
Oct 30 19:21:28.086928 ip-10-0-132-152 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 19:21:28.105429 ip-10-0-132-152 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 19:21:28.105610 ip-10-0-132-152 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 19:21:28.105662 ip-10-0-132-152 systemd[1]: kubelet.service: Consumed 4.773s CPU time.
Oct 30 19:21:28.124522 ip-10-0-132-152 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 19:21:28.187422 ip-10-0-132-152 kubenswrapper[5907]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 19:21:28.187422 ip-10-0-132-152 kubenswrapper[5907]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 19:21:28.187422 ip-10-0-132-152 kubenswrapper[5907]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 19:21:28.187422 ip-10-0-132-152 kubenswrapper[5907]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 19:21:28.189521 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189429    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:21:28.189521 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189520    5907 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:21:28.189521 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189524    5907 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:21:28.189612 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189527    5907 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:21:28.189612 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189531    5907 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:21:28.189612 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189534    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:21:28.189612 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189537    5907 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:21:28.189612 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189539    5907 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:21:28.189612 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189542    5907 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:21:28.189612 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189545    5907 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:21:28.189612 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189547    5907 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:21:28.189612 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189550    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:21:28.189612 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189552    5907 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:21:28.189612 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189555    5907 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:21:28.189612 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189558    5907 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:21:28.189612 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189560    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:21:28.189612 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189563    5907 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:21:28.189612 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189565    5907 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:21:28.189612 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189568    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:21:28.189612 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189570    5907 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:21:28.189612 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189573    5907 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:21:28.189612 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189575    5907 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:21:28.189612 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189579    5907 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:21:28.190133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189581    5907 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:21:28.190133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189584    5907 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:21:28.190133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189587    5907 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:21:28.190133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189597    5907 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:21:28.190133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189600    5907 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:21:28.190133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189603    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:21:28.190133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189605    5907 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:21:28.190133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189608    5907 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:21:28.190133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189610    5907 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:21:28.190133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189613    5907 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:21:28.190133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189616    5907 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:21:28.190133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189619    5907 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:21:28.190133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189622    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:21:28.190133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189627    5907 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:21:28.190133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189631    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:21:28.190133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189634    5907 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:21:28.190133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189637    5907 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:21:28.190133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189640    5907 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:21:28.190133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189643    5907 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:21:28.190611 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189645    5907 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:21:28.190611 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189648    5907 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:21:28.190611 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189651    5907 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:21:28.190611 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189655    5907 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:21:28.190611 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189659    5907 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:21:28.190611 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189662    5907 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:21:28.190611 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189665    5907 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:21:28.190611 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189667    5907 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:21:28.190611 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189669    5907 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:21:28.190611 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189672    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:21:28.190611 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189674    5907 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:21:28.190611 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189677    5907 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:21:28.190611 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189680    5907 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:21:28.190611 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189683    5907 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:21:28.190611 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189685    5907 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:21:28.190611 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189688    5907 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:21:28.190611 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189690    5907 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:21:28.190611 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189693    5907 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:21:28.190611 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189696    5907 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:21:28.191086 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189703    5907 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:21:28.191086 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189706    5907 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:21:28.191086 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189709    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:21:28.191086 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189711    5907 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:21:28.191086 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189714    5907 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:21:28.191086 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189717    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:21:28.191086 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189720    5907 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:21:28.191086 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189722    5907 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:21:28.191086 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189725    5907 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:21:28.191086 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189727    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:21:28.191086 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189730    5907 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:21:28.191086 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189732    5907 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:21:28.191086 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189734    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:21:28.191086 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189737    5907 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:21:28.191086 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189739    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:21:28.191086 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189743    5907 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:21:28.191086 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189745    5907 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:21:28.191086 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189748    5907 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:21:28.191086 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189751    5907 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:21:28.191086 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189753    5907 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:21:28.191597 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189757    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:21:28.191597 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189760    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:21:28.191597 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189762    5907 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:21:28.191597 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189764    5907 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:21:28.191597 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189767    5907 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:21:28.191597 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189769    5907 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:21:28.191597 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189772    5907 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:21:28.191597 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189774    5907 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:21:28.191597 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189777    5907 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:21:28.191597 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189779    5907 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:21:28.191597 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189781    5907 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:21:28.191597 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189784    5907 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:21:28.191597 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189788    5907 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:21:28.191597 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189791    5907 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:21:28.191597 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189793    5907 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:21:28.191597 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189796    5907 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:21:28.191597 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189799    5907 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:21:28.191597 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189803    5907 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:21:28.191597 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189806    5907 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:21:28.191597 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189808    5907 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:21:28.192106 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189811    5907 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:21:28.192106 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189813    5907 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:21:28.192106 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189816    5907 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:21:28.192106 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189819    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:21:28.192106 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.189821    5907 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:21:28.192106 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190662    5907 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:21:28.192106 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190671    5907 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:21:28.192106 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190675    5907 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:21:28.192106 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190677    5907 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:21:28.192106 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190681    5907 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:21:28.192106 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190687    5907 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:21:28.192106 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190690    5907 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:21:28.192106 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190693    5907 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:21:28.192106 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190696    5907 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:21:28.192106 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190698    5907 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:21:28.192106 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190701    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:21:28.192106 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190704    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:21:28.192106 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190706    5907 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:21:28.192106 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190709    5907 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:21:28.192106 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190712    5907 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:21:28.192607 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190714    5907 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:21:28.192607 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190716    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:21:28.192607 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190719    5907 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:21:28.192607 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190722    5907 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:21:28.192607 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190725    5907 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:21:28.192607 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190728    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:21:28.192607 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190730    5907 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:21:28.192607 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190733    5907 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:21:28.192607 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190736    5907 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:21:28.192607 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190738    5907 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:21:28.192607 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190741    5907 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:21:28.192607 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190744    5907 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:21:28.192607 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190746    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:21:28.192607 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190748    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:21:28.192607 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190751    5907 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:21:28.192607 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190753    5907 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:21:28.192607 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190756    5907 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:21:28.192607 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190758    5907 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:21:28.192607 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190761    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:21:28.193076 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190763    5907 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:21:28.193076 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190766    5907 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:21:28.193076 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190768    5907 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:21:28.193076 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190770    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:21:28.193076 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190775    5907 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:21:28.193076 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190779    5907 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:21:28.193076 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190783    5907 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:21:28.193076 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190785    5907 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:21:28.193076 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190788    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:21:28.193076 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190792    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:21:28.193076 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190795    5907 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:21:28.193076 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190797    5907 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:21:28.193076 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190800    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:21:28.193076 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190802    5907 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:21:28.193076 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190805    5907 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:21:28.193076 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190807    5907 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:21:28.193076 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190810    5907 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:21:28.193076 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190812    5907 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:21:28.193076 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190815    5907 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:21:28.193532 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190817    5907 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:21:28.193532 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190820    5907 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:21:28.193532 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190823    5907 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:21:28.193532 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190825    5907 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:21:28.193532 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190828    5907 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:21:28.193532 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190831    5907 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:21:28.193532 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190833    5907 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:21:28.193532 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190836    5907 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:21:28.193532 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190838    5907 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:21:28.193532 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190841    5907 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:21:28.193532 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190843    5907 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:21:28.193532 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190845    5907 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:21:28.193532 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190848    5907 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:21:28.193532 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190850    5907 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:21:28.193532 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190852    5907 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:21:28.193532 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190875    5907 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:21:28.193532 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190878    5907 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:21:28.193532 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190881    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:21:28.193532 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190884    5907 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:21:28.193532 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190887    5907 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:21:28.194066 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190890    5907 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:21:28.194066 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190892    5907 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:21:28.194066 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190895    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:21:28.194066 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190909    5907 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:21:28.194066 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190913    5907 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:21:28.194066 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190915    5907 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:21:28.194066 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190918    5907 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:21:28.194066 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190921    5907 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:21:28.194066 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190923    5907 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:21:28.194066 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190926    5907 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:21:28.194066 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190928    5907 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:21:28.194066 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190931    5907 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:21:28.194066 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190933    5907 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:21:28.194066 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190936    5907 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:21:28.194066 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190939    5907 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:21:28.194066 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190941    5907 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:21:28.194066 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190944    5907 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:21:28.194066 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190947    5907 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:21:28.194066 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190949    5907 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:21:28.194066 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190952    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:21:28.194586 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190954    5907 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:21:28.194586 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190956    5907 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:21:28.194586 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190959    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:21:28.194586 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190962    5907 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:21:28.194586 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190964    5907 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:21:28.194586 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190967    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:21:28.194586 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190969    5907 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:21:28.194586 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190972    5907 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:21:28.194586 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190974    5907 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:21:28.194586 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190977    5907 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:21:28.194586 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190979    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:21:28.194586 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190984    5907 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:21:28.194586 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.190989    5907 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:21:28.194586 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191479    5907 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:21:28.194586 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191485    5907 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:21:28.194586 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191488    5907 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:21:28.194586 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191490    5907 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:21:28.194586 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191493    5907 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:21:28.194586 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191496    5907 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:21:28.195075 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191499    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:21:28.195075 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191502    5907 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:21:28.195075 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191504    5907 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:21:28.195075 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191507    5907 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:21:28.195075 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191509    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:21:28.195075 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191512    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:21:28.195075 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191514    5907 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:21:28.195075 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191517    5907 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:21:28.195075 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191520    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:21:28.195075 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191522    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:21:28.195075 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191526    5907 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:21:28.195075 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191530    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:21:28.195075 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191533    5907 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:21:28.195075 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191535    5907 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:21:28.195075 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191538    5907 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:21:28.195075 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191540    5907 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:21:28.195075 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191543    5907 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:21:28.195075 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191545    5907 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:21:28.195075 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191548    5907 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:21:28.195075 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191550    5907 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:21:28.195688 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191553    5907 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:21:28.195688 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191555    5907 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:21:28.195688 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191558    5907 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:21:28.195688 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191560    5907 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:21:28.195688 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191563    5907 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:21:28.195688 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191566    5907 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:21:28.195688 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191569    5907 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:21:28.195688 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191571    5907 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:21:28.195688 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191573    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:21:28.195688 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191576    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:21:28.195688 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191578    5907 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:21:28.195688 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191583    5907 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:21:28.195688 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191587    5907 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:21:28.195688 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191590    5907 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:21:28.195688 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191593    5907 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:21:28.195688 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191595    5907 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:21:28.195688 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191598    5907 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:21:28.195688 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191601    5907 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:21:28.195688 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191603    5907 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:21:28.196547 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191608    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:21:28.196547 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191610    5907 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:21:28.196547 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191613    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:21:28.196547 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191615    5907 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:21:28.196547 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191620    5907 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:21:28.196547 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191622    5907 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:21:28.196547 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191625    5907 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:21:28.196547 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191627    5907 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:21:28.196547 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191630    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:21:28.196547 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191632    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:21:28.196547 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191635    5907 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:21:28.196547 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191637    5907 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:21:28.196547 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191640    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:21:28.196547 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191643    5907 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:21:28.196547 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191645    5907 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:21:28.196547 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191647    5907 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:21:28.196547 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191650    5907 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:21:28.196547 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191652    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:21:28.196547 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191655    5907 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:21:28.197449 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191658    5907 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:21:28.197449 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191660    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:21:28.197449 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191663    5907 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:21:28.197449 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191665    5907 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:21:28.197449 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191667    5907 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:21:28.197449 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191670    5907 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:21:28.197449 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191672    5907 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:21:28.197449 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191675    5907 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:21:28.197449 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191677    5907 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:21:28.197449 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191680    5907 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:21:28.197449 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191682    5907 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:21:28.197449 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191685    5907 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:21:28.197449 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191688    5907 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:21:28.197449 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191692    5907 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:21:28.197449 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191696    5907 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:21:28.197449 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191700    5907 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:21:28.197449 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191703    5907 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:21:28.197449 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191707    5907 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:21:28.197449 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191709    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:21:28.198340 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191712    5907 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:21:28.198340 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191714    5907 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:21:28.198340 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191717    5907 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:21:28.198340 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191719    5907 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:21:28.198340 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191722    5907 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:21:28.198340 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191725    5907 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:21:28.198340 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191728    5907 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:21:28.198340 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191730    5907 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:21:28.198340 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191732    5907 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:21:28.198340 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191735    5907 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:21:28.198340 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191737    5907 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:21:28.198340 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191740    5907 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:21:28.198340 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191742    5907 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:21:28.198340 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191745    5907 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:21:28.198340 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191748    5907 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:21:28.198340 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191750    5907 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:21:28.198340 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191753    5907 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:21:28.198340 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191755    5907 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:21:28.198340 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191758    5907 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:21:28.198340 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191760    5907 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:21:28.199228 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191763    5907 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:21:28.199228 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191766    5907 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:21:28.199228 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.191768    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:21:28.199228 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.191882    5907 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 19:21:28.199228 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.191890    5907 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 19:21:28.199228 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.191897    5907 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 19:21:28.199228 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.191904    5907 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 19:21:28.199228 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.191908    5907 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 19:21:28.199228 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.191912    5907 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 19:21:28.199228 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.191916    5907 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 19:21:28.199228 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.191921    5907 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 19:21:28.199228 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.191924    5907 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 19:21:28.199228 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.191927    5907 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 19:21:28.199228 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.191931    5907 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 19:21:28.199228 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.191935    5907 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 19:21:28.199228 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.191938    5907 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 19:21:28.199228 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.191941    5907 flags.go:64] FLAG: --cgroup-root=""
Oct 30 19:21:28.199228 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.191943    5907 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 19:21:28.199228 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.191947    5907 flags.go:64] FLAG: --client-ca-file=""
Oct 30 19:21:28.199228 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.191950    5907 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 19:21:28.199228 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.191953    5907 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 19:21:28.199228 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.191956    5907 flags.go:64] FLAG: --cluster-domain=""
Oct 30 19:21:28.199228 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.191959    5907 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 19:21:28.200218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.191962    5907 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 19:21:28.200218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.191965    5907 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 19:21:28.200218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.191969    5907 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 19:21:28.200218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.191973    5907 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 19:21:28.200218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.191976    5907 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 19:21:28.200218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.191980    5907 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 19:21:28.200218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.191983    5907 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 19:21:28.200218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.191986    5907 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 19:21:28.200218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.191989    5907 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 19:21:28.200218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.191992    5907 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 19:21:28.200218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.191995    5907 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 19:21:28.200218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.191997    5907 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 19:21:28.200218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192001    5907 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 19:21:28.200218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192004    5907 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 19:21:28.200218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192007    5907 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 19:21:28.200218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192015    5907 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 19:21:28.200218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192018    5907 flags.go:64] FLAG: --enable-server="true"
Oct 30 19:21:28.200218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192021    5907 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 19:21:28.200218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192026    5907 flags.go:64] FLAG: --event-burst="100"
Oct 30 19:21:28.200218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192030    5907 flags.go:64] FLAG: --event-qps="50"
Oct 30 19:21:28.200218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192033    5907 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 19:21:28.200218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192036    5907 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 19:21:28.200218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192039    5907 flags.go:64] FLAG: --eviction-hard=""
Oct 30 19:21:28.200218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192043    5907 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 19:21:28.201361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192046    5907 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 19:21:28.201361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192049    5907 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 19:21:28.201361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192052    5907 flags.go:64] FLAG: --eviction-soft=""
Oct 30 19:21:28.201361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192055    5907 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 19:21:28.201361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192058    5907 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 19:21:28.201361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192061    5907 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 19:21:28.201361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192064    5907 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 19:21:28.201361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192067    5907 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 19:21:28.201361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192070    5907 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 19:21:28.201361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192072    5907 flags.go:64] FLAG: --feature-gates=""
Oct 30 19:21:28.201361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192077    5907 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 19:21:28.201361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192080    5907 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 19:21:28.201361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192084    5907 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 19:21:28.201361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192087    5907 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 19:21:28.201361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192090    5907 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 19:21:28.201361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192093    5907 flags.go:64] FLAG: --help="false"
Oct 30 19:21:28.201361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192096    5907 flags.go:64] FLAG: --hostname-override="ip-10-0-132-152.ec2.internal"
Oct 30 19:21:28.201361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192099    5907 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 19:21:28.201361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192101    5907 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 19:21:28.201361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192104    5907 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 19:21:28.201361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192107    5907 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 19:21:28.201361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192111    5907 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 19:21:28.201361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192114    5907 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 19:21:28.201361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192118    5907 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 19:21:28.202309 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192121    5907 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 19:21:28.202309 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192124    5907 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 19:21:28.202309 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192127    5907 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 19:21:28.202309 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192131    5907 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 19:21:28.202309 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192134    5907 flags.go:64] FLAG: --kube-reserved=""
Oct 30 19:21:28.202309 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192137    5907 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 19:21:28.202309 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192140    5907 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 19:21:28.202309 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192143    5907 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 19:21:28.202309 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192146    5907 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 19:21:28.202309 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192148    5907 flags.go:64] FLAG: --lock-file=""
Oct 30 19:21:28.202309 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192151    5907 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 19:21:28.202309 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192154    5907 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 19:21:28.202309 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192157    5907 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 19:21:28.202309 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192163    5907 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 19:21:28.202309 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192165    5907 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 19:21:28.202309 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192168    5907 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 19:21:28.202309 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192171    5907 flags.go:64] FLAG: --logging-format="text"
Oct 30 19:21:28.202309 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192174    5907 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 19:21:28.202309 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192177    5907 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 19:21:28.202309 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192180    5907 flags.go:64] FLAG: --manifest-url=""
Oct 30 19:21:28.202309 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192183    5907 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 19:21:28.202309 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192187    5907 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 19:21:28.202309 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192190    5907 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 19:21:28.202309 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192194    5907 flags.go:64] FLAG: --max-pods="110"
Oct 30 19:21:28.202309 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192197    5907 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 19:21:28.203074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192200    5907 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 19:21:28.203074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192203    5907 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 19:21:28.203074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192206    5907 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 19:21:28.203074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192209    5907 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 19:21:28.203074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192212    5907 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 19:21:28.203074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192214    5907 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 19:21:28.203074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192223    5907 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 19:21:28.203074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192225    5907 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 19:21:28.203074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192228    5907 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 19:21:28.203074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192231    5907 flags.go:64] FLAG: --pod-cidr=""
Oct 30 19:21:28.203074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192235    5907 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 19:21:28.203074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192238    5907 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 19:21:28.203074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192241    5907 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 19:21:28.203074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192244    5907 flags.go:64] FLAG: --port="10250"
Oct 30 19:21:28.203074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192247    5907 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 19:21:28.203074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192250    5907 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-064ee3f159afc8f03"
Oct 30 19:21:28.203074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192253    5907 flags.go:64] FLAG: --qos-reserved=""
Oct 30 19:21:28.203074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192256    5907 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 19:21:28.203074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192259    5907 flags.go:64] FLAG: --register-node="true"
Oct 30 19:21:28.203074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192262    5907 flags.go:64] FLAG: --register-with-taints=""
Oct 30 19:21:28.203074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192269    5907 flags.go:64] FLAG: --registry-burst="10"
Oct 30 19:21:28.203074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192272    5907 flags.go:64] FLAG: --registry-qps="5"
Oct 30 19:21:28.203074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192275    5907 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 19:21:28.203074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192278    5907 flags.go:64] FLAG: --reserved-memory=""
Oct 30 19:21:28.203074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192282    5907 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 19:21:28.203074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192284    5907 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 19:21:28.203767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192288    5907 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 19:21:28.203767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192290    5907 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 19:21:28.203767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192293    5907 flags.go:64] FLAG: --runonce="false"
Oct 30 19:21:28.203767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192296    5907 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 19:21:28.203767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192299    5907 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 19:21:28.203767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192302    5907 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 19:21:28.203767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192305    5907 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 19:21:28.203767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192308    5907 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 19:21:28.203767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192311    5907 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 19:21:28.203767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192314    5907 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 19:21:28.203767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192317    5907 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 19:21:28.203767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192319    5907 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 19:21:28.203767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192324    5907 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 19:21:28.203767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192327    5907 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 19:21:28.203767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192329    5907 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 19:21:28.203767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192332    5907 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 19:21:28.203767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192336    5907 flags.go:64] FLAG: --system-cgroups=""
Oct 30 19:21:28.203767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192339    5907 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 19:21:28.203767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192344    5907 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 19:21:28.203767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192347    5907 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 19:21:28.203767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192350    5907 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 19:21:28.203767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192354    5907 flags.go:64] FLAG: --tls-min-version=""
Oct 30 19:21:28.203767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192357    5907 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 19:21:28.203767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192359    5907 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 19:21:28.203767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192362    5907 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 19:21:28.204396 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192365    5907 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 19:21:28.204396 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192368    5907 flags.go:64] FLAG: --v="2"
Oct 30 19:21:28.204396 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192372    5907 flags.go:64] FLAG: --version="false"
Oct 30 19:21:28.204396 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192376    5907 flags.go:64] FLAG: --vmodule=""
Oct 30 19:21:28.204396 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192380    5907 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 19:21:28.204396 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192383    5907 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 19:21:28.204396 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192531    5907 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:21:28.204396 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192536    5907 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:21:28.204396 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192539    5907 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:21:28.204396 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192541    5907 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:21:28.204396 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192544    5907 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:21:28.204396 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192547    5907 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:21:28.204396 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192551    5907 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:21:28.204396 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192556    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:21:28.204396 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192558    5907 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:21:28.204396 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192561    5907 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:21:28.204396 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192564    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:21:28.204396 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192566    5907 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:21:28.204396 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192569    5907 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:21:28.204396 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192572    5907 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:21:28.204396 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192575    5907 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:21:28.204918 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192577    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:21:28.204918 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192580    5907 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:21:28.204918 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192584    5907 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:21:28.204918 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192587    5907 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:21:28.204918 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192589    5907 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:21:28.204918 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192592    5907 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:21:28.204918 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192594    5907 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:21:28.204918 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192597    5907 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:21:28.204918 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192599    5907 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:21:28.204918 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192602    5907 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:21:28.204918 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192605    5907 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:21:28.204918 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192607    5907 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:21:28.204918 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192610    5907 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:21:28.204918 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192613    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:21:28.204918 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192616    5907 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:21:28.204918 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192618    5907 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:21:28.204918 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192621    5907 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:21:28.204918 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192623    5907 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:21:28.204918 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192626    5907 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:21:28.204918 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192628    5907 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:21:28.205526 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192630    5907 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:21:28.205526 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192633    5907 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:21:28.205526 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192635    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:21:28.205526 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192638    5907 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:21:28.205526 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192641    5907 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:21:28.205526 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192643    5907 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:21:28.205526 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192646    5907 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:21:28.205526 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192649    5907 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:21:28.205526 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192651    5907 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:21:28.205526 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192653    5907 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:21:28.205526 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192656    5907 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:21:28.205526 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192659    5907 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:21:28.205526 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192661    5907 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:21:28.205526 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192664    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:21:28.205526 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192667    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:21:28.205526 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192669    5907 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:21:28.205526 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192672    5907 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:21:28.205526 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192674    5907 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:21:28.205526 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192676    5907 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:21:28.205526 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192679    5907 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:21:28.206208 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192681    5907 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:21:28.206208 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192684    5907 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:21:28.206208 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192686    5907 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:21:28.206208 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192689    5907 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:21:28.206208 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192691    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:21:28.206208 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192693    5907 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:21:28.206208 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192696    5907 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:21:28.206208 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192698    5907 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:21:28.206208 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192701    5907 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:21:28.206208 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192703    5907 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:21:28.206208 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192706    5907 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:21:28.206208 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192708    5907 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:21:28.206208 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192710    5907 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:21:28.206208 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192713    5907 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:21:28.206208 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192715    5907 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:21:28.206208 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192717    5907 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:21:28.206208 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192720    5907 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:21:28.206208 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192724    5907 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:21:28.206208 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192728    5907 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:21:28.206730 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192730    5907 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:21:28.206730 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192734    5907 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:21:28.206730 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192736    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:21:28.206730 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192739    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:21:28.206730 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192741    5907 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:21:28.206730 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192744    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:21:28.206730 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192746    5907 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:21:28.206730 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192748    5907 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:21:28.206730 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192751    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:21:28.206730 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192754    5907 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:21:28.206730 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192756    5907 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:21:28.206730 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192758    5907 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:21:28.206730 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192761    5907 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:21:28.206730 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192763    5907 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:21:28.206730 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192765    5907 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:21:28.206730 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192768    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:21:28.206730 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192770    5907 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:21:28.206730 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192773    5907 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:21:28.206730 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192775    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:21:28.207469 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192778    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:21:28.207469 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192780    5907 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:21:28.207469 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192782    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:21:28.207469 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192785    5907 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:21:28.207469 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192787    5907 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:21:28.207469 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192790    5907 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:21:28.207469 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192792    5907 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:21:28.207469 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192794    5907 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:21:28.207469 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192797    5907 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:21:28.207469 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192799    5907 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:21:28.207469 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192801    5907 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:21:28.207469 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192804    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:21:28.207469 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192807    5907 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:21:28.207469 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.192832    5907 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 19:21:28.207469 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192892    5907 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:21:28.207469 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192897    5907 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:21:28.208375 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192900    5907 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:21:28.208375 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192903    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:21:28.208375 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192906    5907 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:21:28.208375 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192909    5907 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:21:28.208375 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192913    5907 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:21:28.208375 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192915    5907 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:21:28.208375 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192918    5907 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:21:28.208375 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192920    5907 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:21:28.208375 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192923    5907 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:21:28.208375 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192925    5907 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:21:28.208375 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192928    5907 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:21:28.208375 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192930    5907 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:21:28.208375 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192933    5907 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:21:28.208375 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192935    5907 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:21:28.208375 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192938    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:21:28.208375 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192940    5907 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:21:28.208375 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192942    5907 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:21:28.208375 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192945    5907 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:21:28.208375 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192947    5907 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:21:28.208375 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192950    5907 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:21:28.209695 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192952    5907 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:21:28.209695 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192954    5907 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:21:28.209695 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192957    5907 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:21:28.209695 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192959    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:21:28.209695 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192961    5907 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:21:28.209695 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192964    5907 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:21:28.209695 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192966    5907 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:21:28.209695 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192969    5907 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:21:28.209695 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192972    5907 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:21:28.209695 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192974    5907 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:21:28.209695 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192977    5907 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:21:28.209695 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192979    5907 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:21:28.209695 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192982    5907 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:21:28.209695 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192984    5907 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:21:28.209695 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192986    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:21:28.209695 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192989    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:21:28.209695 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192991    5907 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:21:28.209695 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192994    5907 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:21:28.209695 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192996    5907 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:21:28.209695 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.192998    5907 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:21:28.210729 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193001    5907 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:21:28.210729 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193005    5907 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:21:28.210729 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193008    5907 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:21:28.210729 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193010    5907 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:21:28.210729 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193013    5907 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:21:28.210729 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193015    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:21:28.210729 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193018    5907 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:21:28.210729 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193020    5907 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:21:28.210729 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193023    5907 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:21:28.210729 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193026    5907 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:21:28.210729 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193028    5907 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:21:28.210729 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193031    5907 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:21:28.210729 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193033    5907 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:21:28.210729 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193036    5907 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:21:28.210729 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193038    5907 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:21:28.210729 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193040    5907 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:21:28.210729 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193043    5907 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:21:28.210729 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193045    5907 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:21:28.210729 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193049    5907 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:21:28.211328 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193053    5907 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:21:28.211328 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193056    5907 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:21:28.211328 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193061    5907 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:21:28.211328 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193063    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:21:28.211328 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193066    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:21:28.211328 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193069    5907 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:21:28.211328 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193071    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:21:28.211328 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193073    5907 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:21:28.211328 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193076    5907 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:21:28.211328 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193079    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:21:28.211328 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193082    5907 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:21:28.211328 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193084    5907 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:21:28.211328 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193087    5907 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:21:28.211328 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193090    5907 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:21:28.211328 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193093    5907 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:21:28.211328 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193096    5907 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:21:28.211328 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193098    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:21:28.211328 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193101    5907 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:21:28.211328 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193103    5907 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:21:28.211917 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193106    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:21:28.211917 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193108    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:21:28.211917 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193111    5907 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:21:28.211917 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193113    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:21:28.211917 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193116    5907 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:21:28.211917 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193119    5907 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:21:28.211917 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193121    5907 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:21:28.211917 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193123    5907 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:21:28.211917 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193126    5907 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:21:28.211917 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193128    5907 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:21:28.211917 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193131    5907 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:21:28.211917 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193133    5907 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:21:28.211917 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193136    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:21:28.211917 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193138    5907 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:21:28.211917 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193141    5907 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:21:28.211917 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193146    5907 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:21:28.211917 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193149    5907 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:21:28.211917 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193152    5907 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:21:28.211917 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193154    5907 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:21:28.211917 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193157    5907 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:21:28.212399 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193162    5907 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:21:28.212399 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193165    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:21:28.212399 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193168    5907 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:21:28.212399 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193171    5907 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:21:28.212399 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193173    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:21:28.212399 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.193176    5907 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:21:28.212399 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.198453    5907 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 19:21:28.212399 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.198472    5907 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 19:21:28.212399 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.198497    5907 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 19:21:28.212399 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.198504    5907 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 19:21:28.212399 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198619    5907 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:21:28.212399 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198628    5907 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:21:28.212399 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198634    5907 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:21:28.212399 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198639    5907 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:21:28.212399 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198644    5907 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:21:28.212399 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198649    5907 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:21:28.212399 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198653    5907 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:21:28.212399 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198658    5907 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:21:28.212399 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198663    5907 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:21:28.212942 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198668    5907 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:21:28.212942 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198673    5907 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:21:28.212942 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198677    5907 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:21:28.212942 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198682    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:21:28.212942 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198687    5907 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:21:28.212942 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198691    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:21:28.212942 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198695    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:21:28.212942 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198701    5907 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:21:28.212942 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198706    5907 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:21:28.212942 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198711    5907 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:21:28.212942 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198716    5907 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:21:28.212942 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198721    5907 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:21:28.212942 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198726    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:21:28.212942 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198730    5907 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:21:28.212942 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198743    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:21:28.212942 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198749    5907 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:21:28.212942 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198753    5907 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:21:28.212942 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198758    5907 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:21:28.212942 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198763    5907 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:21:28.212942 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198767    5907 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:21:28.213418 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198772    5907 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:21:28.213418 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198776    5907 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:21:28.213418 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198781    5907 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:21:28.213418 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198786    5907 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:21:28.213418 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198791    5907 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:21:28.213418 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198798    5907 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:21:28.213418 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198805    5907 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:21:28.213418 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198810    5907 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:21:28.213418 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198816    5907 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:21:28.213418 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198821    5907 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:21:28.213418 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198826    5907 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:21:28.213418 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198830    5907 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:21:28.213418 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198835    5907 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:21:28.213418 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198839    5907 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:21:28.213418 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198844    5907 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:21:28.213418 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198849    5907 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:21:28.213418 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198853    5907 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:21:28.213418 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198879    5907 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:21:28.213418 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198885    5907 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:21:28.213418 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198889    5907 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:21:28.213910 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198894    5907 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:21:28.213910 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198898    5907 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:21:28.213910 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198903    5907 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:21:28.213910 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198909    5907 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:21:28.213910 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198913    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:21:28.213910 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198918    5907 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:21:28.213910 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198922    5907 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:21:28.213910 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198937    5907 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:21:28.213910 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198943    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:21:28.213910 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198948    5907 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:21:28.213910 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198953    5907 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:21:28.213910 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198957    5907 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:21:28.213910 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198962    5907 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:21:28.213910 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198967    5907 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:21:28.213910 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198971    5907 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:21:28.213910 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198976    5907 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:21:28.213910 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198980    5907 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:21:28.213910 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198984    5907 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:21:28.213910 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198989    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:21:28.214371 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.198996    5907 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:21:28.214371 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199002    5907 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:21:28.214371 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199008    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:21:28.214371 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199013    5907 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:21:28.214371 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199018    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:21:28.214371 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199022    5907 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:21:28.214371 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199027    5907 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:21:28.214371 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199032    5907 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:21:28.214371 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199037    5907 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:21:28.214371 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199041    5907 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:21:28.214371 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199046    5907 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:21:28.214371 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199051    5907 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:21:28.214371 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199056    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:21:28.214371 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199060    5907 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:21:28.214371 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199064    5907 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:21:28.214371 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199069    5907 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:21:28.214371 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199074    5907 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:21:28.214371 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199079    5907 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:21:28.214371 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199083    5907 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:21:28.214931 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199088    5907 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:21:28.214931 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199093    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:21:28.214931 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199105    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:21:28.214931 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199110    5907 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:21:28.214931 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199115    5907 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:21:28.214931 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199120    5907 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:21:28.214931 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199124    5907 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:21:28.214931 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199129    5907 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:21:28.214931 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199134    5907 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:21:28.214931 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199139    5907 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:21:28.214931 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199143    5907 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:21:28.214931 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199147    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:21:28.214931 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199151    5907 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:21:28.214931 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199156    5907 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:21:28.214931 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199161    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:21:28.214931 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199165    5907 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:21:28.214931 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199170    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:21:28.214931 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199174    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:21:28.214931 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199179    5907 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:21:28.215702 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.199221    5907 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 19:21:28.215702 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199538    5907 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:21:28.215702 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199548    5907 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:21:28.215702 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199563    5907 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:21:28.215702 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199568    5907 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:21:28.215702 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199573    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:21:28.215702 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199578    5907 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:21:28.215702 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199583    5907 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:21:28.215702 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199587    5907 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:21:28.215702 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199592    5907 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:21:28.215702 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199597    5907 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:21:28.215702 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199602    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:21:28.215702 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199606    5907 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:21:28.215702 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199613    5907 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:21:28.215702 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199620    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:21:28.216341 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199627    5907 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:21:28.216341 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199640    5907 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:21:28.216341 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199646    5907 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:21:28.216341 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199650    5907 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:21:28.216341 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199655    5907 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:21:28.216341 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199660    5907 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:21:28.216341 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199665    5907 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:21:28.216341 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199669    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:21:28.216341 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199674    5907 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:21:28.216341 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199678    5907 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:21:28.216341 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199683    5907 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:21:28.216341 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199688    5907 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:21:28.216341 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199692    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:21:28.216341 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199697    5907 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:21:28.216341 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199702    5907 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:21:28.216341 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199706    5907 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:21:28.216341 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199711    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:21:28.216341 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199716    5907 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:21:28.216341 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199720    5907 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:21:28.216341 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199727    5907 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:21:28.217074 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199733    5907 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:21:28.217074 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199737    5907 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:21:28.217074 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199742    5907 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:21:28.217074 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199747    5907 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:21:28.217074 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199751    5907 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:21:28.217074 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199756    5907 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:21:28.217074 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199761    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:21:28.217074 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199766    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:21:28.217074 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199770    5907 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:21:28.217074 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199774    5907 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:21:28.217074 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199779    5907 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:21:28.217074 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199784    5907 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:21:28.217074 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199788    5907 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:21:28.217074 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199793    5907 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:21:28.217074 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199806    5907 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:21:28.217074 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199811    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:21:28.217074 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199815    5907 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:21:28.217074 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199820    5907 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:21:28.217074 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199825    5907 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:21:28.217074 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199830    5907 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:21:28.217679 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199834    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:21:28.217679 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199839    5907 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:21:28.217679 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199843    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:21:28.217679 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199848    5907 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:21:28.217679 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199852    5907 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:21:28.217679 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199875    5907 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:21:28.217679 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199880    5907 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:21:28.217679 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199885    5907 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:21:28.217679 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199889    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:21:28.217679 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199894    5907 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:21:28.217679 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199899    5907 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:21:28.217679 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199905    5907 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:21:28.217679 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199910    5907 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:21:28.217679 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199915    5907 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:21:28.217679 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199920    5907 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:21:28.217679 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199924    5907 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:21:28.217679 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199928    5907 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:21:28.217679 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199933    5907 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:21:28.217679 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199938    5907 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:21:28.217679 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199944    5907 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:21:28.218231 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199949    5907 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:21:28.218231 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199954    5907 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:21:28.218231 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199958    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:21:28.218231 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199963    5907 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:21:28.218231 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199967    5907 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:21:28.218231 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199971    5907 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:21:28.218231 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.199976    5907 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:21:28.218231 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.200041    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:21:28.218231 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.200703    5907 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:21:28.218231 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.200712    5907 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:21:28.218231 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.200717    5907 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:21:28.218231 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.200722    5907 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:21:28.218231 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.200728    5907 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:21:28.218231 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.200733    5907 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:21:28.218231 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.200738    5907 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:21:28.218231 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.200743    5907 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:21:28.218231 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.200748    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:21:28.218231 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.200752    5907 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:21:28.218231 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.200757    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:21:28.218231 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.200762    5907 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:21:28.218705 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.200767    5907 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:21:28.218705 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.200777    5907 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:21:28.218705 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.200782    5907 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:21:28.218705 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.200788    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:21:28.218705 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.200792    5907 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:21:28.218705 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.200797    5907 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:21:28.218705 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.200802    5907 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:21:28.218705 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.200806    5907 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:21:28.218705 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.200811    5907 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:21:28.218705 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.200816    5907 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:21:28.218705 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.200821    5907 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:21:28.218705 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.200829    5907 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:21:28.218705 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.200922    5907 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 19:21:28.218705 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201093    5907 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:21:28.218705 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201103    5907 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:21:28.218705 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201108    5907 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:21:28.219133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201113    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:21:28.219133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201118    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:21:28.219133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201123    5907 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:21:28.219133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201128    5907 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:21:28.219133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201176    5907 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:21:28.219133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201182    5907 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:21:28.219133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201187    5907 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:21:28.219133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201192    5907 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:21:28.219133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201204    5907 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:21:28.219133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201208    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:21:28.219133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201213    5907 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:21:28.219133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201218    5907 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:21:28.219133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201224    5907 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:21:28.219133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201229    5907 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:21:28.219133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201234    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:21:28.219133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201238    5907 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:21:28.219133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201243    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:21:28.219133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201248    5907 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:21:28.219133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201253    5907 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:21:28.219133 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201259    5907 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:21:28.219999 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201263    5907 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:21:28.219999 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201273    5907 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:21:28.219999 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201278    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:21:28.219999 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201283    5907 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:21:28.219999 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201288    5907 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:21:28.219999 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201294    5907 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:21:28.219999 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201298    5907 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:21:28.219999 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201306    5907 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:21:28.219999 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201311    5907 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:21:28.219999 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201316    5907 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:21:28.219999 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201320    5907 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:21:28.219999 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201325    5907 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:21:28.219999 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201329    5907 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:21:28.219999 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201338    5907 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:21:28.219999 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201343    5907 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:21:28.219999 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201348    5907 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:21:28.219999 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201353    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:21:28.219999 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201413    5907 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:21:28.219999 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201420    5907 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:21:28.219999 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201426    5907 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:21:28.220694 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201431    5907 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:21:28.220694 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201436    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:21:28.220694 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201448    5907 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:21:28.220694 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201453    5907 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:21:28.220694 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201458    5907 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:21:28.220694 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201463    5907 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:21:28.220694 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201473    5907 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:21:28.220694 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201478    5907 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:21:28.220694 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201483    5907 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:21:28.220694 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201487    5907 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:21:28.220694 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201492    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:21:28.220694 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201497    5907 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:21:28.220694 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201502    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:21:28.220694 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201507    5907 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:21:28.220694 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201512    5907 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:21:28.220694 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201517    5907 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:21:28.220694 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201522    5907 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:21:28.220694 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201527    5907 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:21:28.220694 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201532    5907 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:21:28.220694 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201542    5907 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:21:28.221412 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201548    5907 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:21:28.221412 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201553    5907 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:21:28.221412 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201558    5907 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:21:28.221412 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201563    5907 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:21:28.221412 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201568    5907 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:21:28.221412 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201573    5907 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:21:28.221412 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201578    5907 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:21:28.221412 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201587    5907 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:21:28.221412 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201594    5907 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:21:28.221412 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201599    5907 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:21:28.221412 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201665    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:21:28.221412 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201757    5907 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:21:28.221412 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201810    5907 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:21:28.221412 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201814    5907 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:21:28.221412 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201818    5907 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:21:28.221412 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201821    5907 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:21:28.221412 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201824    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:21:28.221412 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201826    5907 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:21:28.221412 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201832    5907 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:21:28.222001 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201837    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:21:28.222001 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201841    5907 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:21:28.222001 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201845    5907 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:21:28.222001 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201848    5907 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:21:28.222001 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201850    5907 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:21:28.222001 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201853    5907 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:21:28.222001 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201873    5907 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:21:28.222001 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201878    5907 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:21:28.222001 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201883    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:21:28.222001 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201886    5907 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:21:28.222001 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201889    5907 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:21:28.222001 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201891    5907 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:21:28.222001 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201896    5907 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:21:28.222001 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201899    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:21:28.222001 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201902    5907 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:21:28.222001 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201905    5907 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:21:28.222001 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201907    5907 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:21:28.222001 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201910    5907 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:21:28.222001 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201912    5907 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:21:28.222001 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201915    5907 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:21:28.222797 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201917    5907 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:21:28.222797 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201923    5907 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:21:28.222797 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201925    5907 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:21:28.222797 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:28.201928    5907 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:21:28.222797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.202044    5907 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 19:21:28.222797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.203044    5907 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 19:21:28.222797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.203102    5907 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 19:21:28.222797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.203283    5907 server.go:1014] "Starting client certificate rotation"
Oct 30 19:21:28.222797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.203425    5907 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 19:21:28.222797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.203475    5907 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 19:15:17 +0000 UTC" deadline="2028-06-18 06:53:26.191875697 +0000 UTC"
Oct 30 19:21:28.222797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.203507    5907 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14315h31m57.988370363s"
Oct 30 19:21:28.222797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.203765    5907 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 19:21:28.222797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.204998    5907 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 19:21:28.222797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.206373    5907 log.go:25] "Validated CRI v1 runtime API"
Oct 30 19:21:28.222797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.210098    5907 log.go:25] "Validated CRI v1 image API"
Oct 30 19:21:28.222797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.211392    5907 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 19:21:28.223231 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.213778    5907 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8d6b887f-5690-45d6-972e-64046250ab11:/dev/nvme0n1p4 f5ae08bf-ff2d-4ff4-b874-94da9c7dd4dd:/dev/nvme0n1p3]
Oct 30 19:21:28.223295 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.213798    5907 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/02f8343e26b330af5d55a4da95611a92cafd596b71d3f7a2b502f1d64f3f2e6d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/02f8343e26b330af5d55a4da95611a92cafd596b71d3f7a2b502f1d64f3f2e6d/userdata/shm major:0 minor:254 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1ecb2762907f7b9889572ec183e48157056de637daf0bd21003cc49cb96208ef/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1ecb2762907f7b9889572ec183e48157056de637daf0bd21003cc49cb96208ef/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2d7437ffa1bf640b1e1e7c6bf743f745578a07f4ca63d213d6bf15dc27281aea/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2d7437ffa1bf640b1e1e7c6bf743f745578a07f4ca63d213d6bf15dc27281aea/userdata/shm major:0 minor:204 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/399dc2728fd6bfe75e8e8f5ea5a208feb42abca5755c9ec2323bdc27e2afc8a5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/399dc2728fd6bfe75e8e8f5ea5a208feb42abca5755c9ec2323bdc27e2afc8a5/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3e2295a83987368051b0ecf5c6b265d334ed32ee0f27fd4214a29d995ad7fd9f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3e2295a83987368051b0ecf5c6b265d334ed32ee0f27fd4214a29d995ad7fd9f/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/454d4e22040463e0ef128cd01598ac2291d8257d485ab2493387e223af92cf71/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/454d4e22040463e0ef128cd01598ac2291d8257d485ab2493387e223af92cf71/userdata/shm major:0 minor:367 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/60ebcfd6f800063ca62814f9d5037ff17ab8021bb25a20a9fb23019628c36457/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/60ebcfd6f800063ca62814f9d5037ff17ab8021bb25a20a9fb23019628c36457/userdata/shm major:0 minor:504 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6ac107b4bbb8ab01e09738c400166c19b041992f42a2ea5a87e53022c6317bd2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6ac107b4bbb8ab01e09738c400166c19b041992f42a2ea5a87e53022c6317bd2/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/79af4843b7a47f58635d1ee037528e4872d865f5a232bb6cbe3589c670b5197d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/79af4843b7a47f58635d1ee037528e4872d865f5a232bb6cbe3589c670b5197d/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/88bd605a5c6886f91bc53cf97ae5477a38e46ac2fb529ef77ac3fe8cb5b6cdfd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/88bd605a5c6886f91bc53cf97ae5477a38e46ac2fb529ef77ac3fe8cb5b6cdfd/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8e6694fb5f64111134a16b0c70a9d467732ebe834c694bb55f0c657b98a0c1fb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8e6694fb5f64111134a16b0c70a9d467732ebe834c694bb55f0c657b98a0c1fb/userdata/shm major:0 minor:81 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8fe73d8245fc2351fe48fe3f797c40b32b675cb0d0242e738ed1856a41b38627/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8fe73d8245fc2351fe48fe3f797c40b32b675cb0d0242e738ed1856a41b38627/userdata/shm major:0 minor:456 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/90cf8b7e2c8a82be30790ee5f04c3379be42be441410a7ed654100bd85592c48/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/90cf8b7e2c8a82be30790ee5f04c3379be42be441410a7ed654100bd85592c48/userdata/shm major:0 minor:506 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9653fcfafb20e3486e0d7dff2ce47bd26c384f2b0b50f8f065899a174fc3c0b6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9653fcfafb20e3486e0d7dff2ce47bd26c384f2b0b50f8f065899a174fc3c0b6/userdata/shm major:0 minor:355 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/990f36d93789ddcfa3d80eae721de26344c8e54c43783b60e92e3128c3a8d2ae/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/990f36d93789ddcfa3d80eae721de26344c8e54c43783b60e92e3128c3a8d2ae/userdata/shm major:0 minor:280 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9f818a927ce3a7d5f3fe85edc4285eb3f0932cc7b6bfc950eec0956ed2e70d7f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9f818a927ce3a7d5f3fe85edc4285eb3f0932cc7b6bfc950eec0956ed2e70d7f/userdata/shm major:0 minor:231 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a7ab6ba877af77a68f6d36be8910e84f896e34840a6913fa1266f5a943f91917/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a7ab6ba877af77a68f6d36be8910e84f896e34840a6913fa1266f5a943f91917/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ab80f0926c3314624b09b3a9706dc0fcd4de7fd2272425a7b5e1cc8744e85053/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ab80f0926c3314624b09b3a9706dc0fcd4de7fd2272425a7b5e1cc8744e85053/userdata/shm major:0 minor:331 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/bebff447d4392a1b68f1a0d6050ed163c27381784c1b08ee6f917e60a37309f0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/bebff447d4392a1b68f1a0d6050ed163c27381784c1b08ee6f917e60a37309f0/userdata/shm major:0 minor:438 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c37476d3d2fccb3f1018e0edaaeacc5cacdba55633f63c064597d568e0345fa6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c37476d3d2fccb3f1018e0edaaeacc5cacdba55633f63c064597d568e0345fa6/userdata/shm major:0 minor:206 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c97b572cfac5edb55782195f4e1400e2bbec702087499bfdac85303de5ba47a5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c97b572cfac5edb55782195f4e1400e2bbec702087499bfdac85303de5ba47a5/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d007f9d11dde4a56571ed3b2441f4f3d884c8bfeb549eecd1a362a77805a1420/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d007f9d11dde4a56571ed3b2441f4f3d884c8bfeb549eecd1a362a77805a1420/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e6f51ee7798f1661bfe55e1c8b7ce3927025c79b69d52572c9c29603d43dcf53/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e6f51ee7798f1661bfe55e1c8b7ce3927025c79b69d52572c9c29603d43dcf53/userdata/shm major:0 minor:501 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ebe32d9b522ba42b347de33cfc25c86867ad907ece79d0455ede7a3820a1c32d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ebe32d9b522ba42b347de33cfc25c86867ad907ece79d0455ede7a3820a1c32d/userdata/shm major:0 minor:265 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f9f6d62c512fcf71a180244d70043254b9c542a889ca7b3feecd9db2ef1a70d2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f9f6d62c512fcf71a180244d70043254b9c542a889ca7b3feecd9db2ef1a70d2/userdata/shm major:0 minor:263 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fe16d935efdb60f47b80f91c0cd9440e9da8c1aa652bdf06a543d6ccec9b3708/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fe16d935efdb60f47b80f91c0cd9440e9da8c1aa652bdf06a543d6ccec9b3708/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1ceff922-50ab-4e0f-b113-03719eb40bc9/volumes/kubernetes.io~projected/kube-api-access-skgdt:{mountpoint:/var/lib/kubelet/pods/1ceff922-50ab-4e0f-b113-03719eb40bc9/volumes/kubernetes.io~projected/kube-api-access-skgdt major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1fda3255-2b22-40a0-870b-7b940977e197/volumes/kubernetes.io~projected/kube-api-access-5jdxg:{mountpoint:/var/lib/kubelet/pods/1fda3255-2b22-40a0-870b-7b940977e197/volumes/kubernetes.io~projected/kube-api-access-5jdxg major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1ffad3bc-f14e-4132-b5ee-3ec84d009252/volumes/kubernetes.io~projected/kube-api-access-mdsrm:{mountpoint:/var/lib/kubelet/pods/1ffad3bc-f14e-4132-b5ee-3ec84d009252/volumes/kubernetes.io~projected/kube-api-access-mdsrm major:0 minor:251 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1ffad3bc-f14e-4132-b5ee-3ec84d009252/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/1ffad3bc-f14e-4132-b5ee-3ec84d009252/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config major:0 minor:247 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1ffad3bc-f14e-4132-b5ee-3ec84d009252/volumes/kubernetes.io~secret/kube-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/1ffad3bc-f14e-4132-b5ee-3ec84d009252/volumes/kubernetes.io~secret/kube-state-metrics-tls major:0 minor:248 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/23ec668e-b2fb-471f-a7c7-13095a3dcf7f/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/23ec668e-b2fb-471f-a7c7-13095a3dcf7f/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/23ec668e-b2fb-471f-a7c7-13095a3dcf7f/volumes/kubernetes.io~projected/kube-api-access-5dls7:{mountpoint:/var/lib/kubelet/pods/23ec668e-b2fb-471f-a7c7-13095a3dcf7f/volumes/kubernetes.io~projected/kube-api-access-5dls7 major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/23ec668e-b2fb-471f-a7c7-13095a3dcf7f/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/23ec668e-b2fb-471f-a7c7-13095a3dcf7f/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2c840c6b-6992-4f29-9023-a0ad99fdc543/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/2c840c6b-6992-4f29-9023-a0ad99fdc543/volumes/kubernetes.io~secret/agent-certs major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2ca12196-1929-4ece-a076-d36527b95ff1/volumes/kubernetes.io~projected/kube-api-access-slnw6:{mountpoint:/var/lib/kubelet/pods/2ca12196-1929-4ece-a076-d36527b95ff1/volumes/kubernetes.io~projected/kube-api-access-slnw6 major:0 minor:354 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2ca12196-1929-4ece-a076-d36527b95ff1/volumes/kubernetes.io~secret/client-ca-bundle:{mountpoint:/var/lib/kubelet/pods/2ca12196-1929-4ece-a076-d36527b95ff1/volumes/kubernetes.io~secret/client-ca-bundle major:0 minor:347 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2ca12196-1929-4ece-a076-d36527b95ff1/volumes/kubernetes.io~secret/secret-metrics-server-client-certs:{mountpoint:/var/lib/kubelet/pods/2ca12196-1929-4ece-a076-d36527b95ff1/volumes/kubernetes.io~secret/secret-metrics-server-client-certs major:0 minor:353 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2ca12196-1929-4ece-a076-d36527b95ff1/volumes/kubernetes.io~secret/secret-metrics-server-tls:{mountpoint:/var/lib/kubelet/pods/2ca12196-1929-4ece-a076-d36527b95ff1/volumes/kubernetes.io~secret/secret-metrics-server-tls major:0 minor:352 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3b05813e-ce3d-4e71-a64e-7417997c2988/volumes/kubernetes.io~projected/kube-api-access-jqrkv:{mountpoint:/var/lib/kubelet/pods/3b05813e-ce3d-4e71-a64e-7417997c2988/volumes/kubernetes.io~projected/kube-api-access-jqrkv major:0 minor:330 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3b05813e-ce3d-4e71-a64e-7417997c2988/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/3b05813e-ce3d-4e71-a64e-7417997c2988/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:329 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3b05813e-ce3d-4e71-a64e-7417997c2988/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/3b05813e-ce3d-4e71-a64e-7417997c2988/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy major:0 minor:326 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3b05813e-ce3d-4e71-a64e-7417997c2988/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics:{mountpoint:/var/lib/kubelet/pods/3b05813e-ce3d-4e71-a64e-7417997c2988/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics major:0 minor:327 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3b05813e-ce3d-4e71-a64e-7417997c2988/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules:{mountpoint:/var/lib/kubelet/pods/3b05813e-ce3d-4e71-a64e-7417997c2988/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules major:0 minor:328 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3b05813e-ce3d-4e71-a64e-7417997c2988/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/3b05813e-ce3d-4e71-a64e-7417997c2988/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web major:0 minor:321 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3b05813e-ce3d-4e71-a64e-7417997c2988/volumes/kubernetes.io~secret/secret-thanos-querier-tls:{mountpoint:/var/lib/kubelet/pods/3b05813e-ce3d-4e71-a64e-7417997c2988/volumes/kubernetes.io~secret/secret-thanos-querier-tls major:0 minor:325 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volume-subpaths/cluster-tls-config/alertmanager/10:{mountpoint:/var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volume-subpaths/cluster-tls-config/alertmanager/10 major:0 minor:301 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volume-subpaths/web-config/alertmanager/9:{mountpoint:/var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volume-subpaths/web-config/alertmanager/9 major:0 minor:297 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volume-subpaths/web-config/config-reloader/6:{mountpoint:/var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volume-subpaths/web-config/config-reloader/6 major:0 minor:297 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volume-subpaths/web-config/init-config-reloader/6:{mountpoint:/var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volume-subpaths/web-config/init-config-reloader/6 major:0 minor:297 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volumes/kubernetes.io~empty-dir/config-out major:0 minor:296 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volumes/kubernetes.io~projected/kube-api-access-6vlf5:{mountpoint:/var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volumes/kubernetes.io~projected/kube-api-access-6vlf5 major:0 minor:303 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volumes/kubernetes.io~projected/tls-assets major:0 minor:292 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volumes/kubernetes.io~secret/cluster-tls-config:{mountpoint:/var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volumes/kubernetes.io~secret/cluster-tls-config major:0 minor:301 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volumes/kubernetes.io~secret/config-volume:{mountpoint:/var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volumes/kubernetes.io~secret/config-volume major:0 minor:298 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy major:0 minor:295 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric:{mountpoint:/var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric major:0 minor:299 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web major:0 minor:300 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volumes/kubernetes.io~secret/secret-alertmanager-main-tls:{mountpoint:/var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volumes/kubernetes.io~secret/secret-alertmanager-main-tls major:0 minor:302 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volumes/kubernetes.io~secret/web-config major:0 minor:297 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/41ff5ee2-70a2-4baf-af4e-00e6575c2e99/volumes/kubernetes.io~projected/kube-api-access-rg2rm:{mountpoint:/var/lib/kubelet/pods/41ff5ee2-70a2-4baf-af4e-00e6575c2e99/volumes/kubernetes.io~projected/kube-api-access-rg2rm major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/41ff5ee2-70a2-4baf-af4e-00e6575c2e99/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/41ff5ee2-70a2-4baf-af4e-00e6575c2e99/volumes/kubernetes.io~secret/metrics-certs major:0 minor:500 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/44b9fc01-6837-46cd-bf01-40cb8fcb90f5/volumes/kubernetes.io~projected/kube-api-access-6klk6:{mountpoint:/var/lib/kubelet/pods/44b9fc01-6837-46cd-bf01-40cb8fcb90f5/volumes/kubernetes.io~projected/kube-api-access-6klk6 major:0 minor:230 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/44b9fc01-6837-46cd-bf01-40cb8fcb90f5/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/44b9fc01-6837-46cd-bf01-40cb8fcb90f5/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:229 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/44b9fc01-6837-46cd-bf01-40cb8fcb90f5/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/44b9fc01-6837-46cd-bf01-40cb8fcb90f5/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4b70ef66-ed5b-4917-8bca-36757fee37e3/volumes/kubernetes.io~projected/kube-api-access-mvcz2:{mountpoint:/var/lib/kubelet/pods/4b70ef66-ed5b-4917-8bca-36757fee37e3/volumes/kubernetes.io~projected/kube-api-access-mvcz2 major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4b70ef66-ed5b-4917-8bca-36757fee37e3/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/4b70ef66-ed5b-4917-8bca-36757fee37e3/volumes/kubernetes.io~secret/metrics-tls major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56/volumes/kubernetes.io~projected/kube-api-access-vv49j:{mountpoint:/var/lib/kubelet/pods/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56/volumes/kubernetes.io~projected/kube-api-access-vv49j major:0 minor:436 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:430 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:435 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/57769216-dd32-4edc-8e73-b4873caa041d/volumes/kubernetes.io~projected/kube-api-access-4rvqr:{mountpoint:/var/lib/kubelet/pods/57769216-dd32-4edc-8e73-b4873caa041d/volumes/kubernetes.io~projected/kube-api-access-4rvqr major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/725d2cce-2a75-426f-8522-d1ddfbabe1b6/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/725d2cce-2a75-426f-8522-d1ddfbabe1b6/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:496 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8161e358-9105-4d99-88c1-98c17db09674/volumes/kubernetes.io~projected/kube-api-access-4hcrs:{mountpoint:/var/lib/kubelet/pods/8161e358-9105-4d99-88c1-98c17db09674/volumes/kubernetes.io~projected/kube-api-access-4hcrs major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9a1d2cec-f539-4106-a185-3134a3df747c/volumes/kubernetes.io~projected/kube-api-access-8k9b2:{mountpoint:/var/lib/kubelet/pods/9a1d2cec-f539-4106-a185-3134a3df747c/volumes/kubernetes.io~projected/kube-api-access-8k9b2 major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9a1d2cec-f539-4106-a185-3134a3df747c/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/9a1d2cec-f539-4106-a185-3134a3df747c/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a4ed12fa-0026-4bf8-a99c-83b04caac774/volumes/kubernetes.io~projected/kube-api-access-rd95d:{mountpoint:/var/lib/kubelet/pods/a4ed12fa-0026-4bf8-a99c-83b04caac774/volumes/kubernetes.io~projected/kube-api-access-rd95d major:0 minor:250 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a4ed12fa-0026-4bf8-a99c-83b04caac774/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/a4ed12fa-0026-4bf8-a99c-83b04caac774/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config major:0 minor:243 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a4ed12fa-0026-4bf8-a99c-83b04caac774/volumes/kubernetes.io~secret/openshift-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/a4ed12fa-0026-4bf8-a99c-83b04caac774/volumes/kubernetes.io~secret/openshift-state-metrics-tls major:0 minor:249 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a8635205-1f61-4a17-bc1f-7876c5536f9c/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/a8635205-1f61-4a17-bc1f-7876c5536f9c/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a8635205-1f61-4a17-bc1f-7876c5536f9c/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/a8635205-1f61-4a17-bc1f-7876c5536f9c/volumes/kubernetes.io~empty-dir/tmp major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a8635205-1f61-4a17-bc1f-7876c5536f9c/volumes/kubernetes.io~projected/kube-api-access-txjsq:{mountpoint:/var/lib/kubelet/pods/a8635205-1f61-4a17-bc1f-7876c5536f9c/volumes/kubernetes.io~projected/kube-api-access-txjsq major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/aa367dd7-da29-424e-9acc-7706f4377494/volumes/kubernetes.io~projected/kube-api-access-s9wwz:{mountpoint:/var/lib/kubelet/pods/aa367dd7-da29-424e-9acc-7706f4377494/volumes/kubernetes.io~projected/kube-api-access-s9wwz major:0 minor:358 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/aa367dd7-da29-424e-9acc-7706f4377494/volumes/kubernetes.io~secret/monitoring-plugin-cert:{mountpoint:/var/lib/kubelet/pods/aa367dd7-da29-424e-9acc-7706f4377494/volumes/kubernetes.io~secret/monitoring-plugin-cert major:0 minor:357 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b41606bf-e8ca-4fc6-8805-b7461187d65e/volumes/kubernetes.io~projected/kube-api-access-bln4w:{mountpoint:/var/lib/kubelet/pods/b41606bf-e8ca-4fc6-8805-b7461187d65e/volumes/kubernetes.io~projected/kube-api-access-bln4w major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ba4ee546-c9c1-4573-b11f-6ba0621972d1/volumes/kubernetes.io~projected/kube-api-access-rwf55:{mountpoint:/var/lib/kubelet/pods/ba4ee546-c9c1-4573-b11f-6ba0621972d1/volumes/kubernetes.io~projected/kube-api-access-rwf55 major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c59e8b44-50b7-4a50-af6a-1bb9a7547a56/volumes/kubernetes.io~projected/kube-api-access-t5c8j:{mountpoint:/var/lib/kubelet/pods/c59e8b44-50b7-4a50-af6a-1bb9a7547a56/volumes/kubernetes.io~projected/kube-api-access-t5c8j major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c59e8b44-50b7-4a50-af6a-1bb9a7547a56/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/c59e8b44-50b7-4a50-af6a-1bb9a7547a56/volumes/kubernetes.io~secret/cert major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cd444ba1-edf1-49aa-a0cf-b863c93e60be/volumes/kubernetes.io~projected/kube-api-access-c2xwk:{mountpoint:/var/lib/kubelet/pods/cd444ba1-edf1-49aa-a0cf-b863c93e60be/volumes/kubernetes.io~projected/kube-api-access-c2xwk major:0 minor:503 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volume-subpaths/web-config/config-reloader/2:{mountpoint:/var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volume-subpaths/web-config/config-reloader/2 major:0 minor:446 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volume-subpaths/web-config/init-config-reloader/2:{mountpoint:/var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volume-subpaths/web-config/init-config-reloader/2 major:0 minor:446 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volume-subpaths/web-config/prometheus/15:{mountpoint:/var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volume-subpaths/web-config/prometheus/15 major:0 minor:446 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~empty-dir/config-out major:0 minor:451 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~projected/kube-api-access-pqn8c:{mountpoint:/var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~projected/kube-api-access-pqn8c major:0 minor:455 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~projected/tls-assets major:0 minor:449 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~secret/config:{mountpoint:/var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~secret/config major:0 minor:445 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:454 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~secret/secret-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~secret/secret-kube-rbac-proxy major:0 minor:448 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~secret/secret-metrics-client-certs:{mountpoint:/var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~secret/secret-metrics-client-certs major:0 minor:452 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web major:0 minor:447 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls:{mountpoint:/var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls major:0 minor:440 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls:{mountpoint:/var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls major:0 minor:450 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file:{mountpoint:/var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file major:0 minor:453 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~secret/web-config major:0 minor:446 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f1622f2e-319b-4259-b314-a4178b7875f0/volumes/kubernetes.io~projected/kube-api-access-wmftk:{mountpoint:/var/lib/kubelet/pods/f1622f2e-319b-4259-b314-a4178b7875f0/volumes/kubernetes.io~projected/kube-api-access-wmftk major:0 minor:259 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f1622f2e-319b-4259-b314-a4178b7875f0/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/f1622f2e-319b-4259-b314-a4178b7875f0/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:256 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f1622f2e-319b-4259-b314-a4178b7875f0/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/f1622f2e-319b-4259-b314-a4178b7875f0/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:257 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/1cb101dd4b58f257bef46562320c894c3b44005a2b0539fea69c3244848b6a60/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/ab431e2fc70fb1e15d2fc224d77c586b187d223136bfcb381f0ce5bfc59d4211/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/a5083918688184483f6c37cd79020f4290c30fc579fdad98e91ff324e14ed91c/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/c2a3d7e07eeaa9261fb8380b2e5cc06e7c506ea963959ed70abc6dc0b3f71a81/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/a54e24c6c1d5bfec4ef5de737ad218b5b9b95d0ed6b0b9c5ec62ce74c652a064/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/bd52473c5592fdd469ca4d3c27c26e1f8de0709cd8a583f4504b87729c019bd7/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/e8beb772446dc2ed883647e48380f0638ab2413f2b934ccd52bacccca0dd10b1/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/4edc7c343529a6e23ac1a63184e035a24cbfc193a275839cae1f006091ed5856/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/5ae0907c3fc9419b5a71100ba5cf8664037cec9511a74cf7fc1031cf4d89be41/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/8a845378d7b53d520d659d238aed43f311c5c18470028f3ca1fffb9e71da8f97/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-142:{mountpoint:/var/lib/containers/storage/overlay/13481e9ecdfce3d2f7024489fa2abdf927c9994da03e4fbeb3ae206eca4e478c/merged major:0 minor:142 fsType:overlay blockSize:0} overlay_0-144:{mountpoint:/var/lib/containers/storage/overlay/dca6c779614e94f9fb8f9730b73517e91b958873da692451490a9cb66708b90d/merged major:0 minor:144 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/4c8d23aa1931b5fcca7e740d026dfa650d46fa8a10321c40f932185144f4c439/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/bcc55290897f1e95a3d54b08fe729d07e4052a2fce0354d00dc4feb93c3b6288/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-161:{mountpoint:/var/lib/containers/storage/overlay/f554ff3e6e8a59bb7d1852e1ab11349a496dbf038b85b18e93c57168a81281d9/merged major:0 minor:161 fsType:overlay blockSize:0} overlay_0-164:{mountpoint:/var/lib/containers/storage/overlay/6b8ac83f07bc69cecfb115b0eb6b4cc6903f5dfb1574ecc34f04ad8f60404efd/merged major:0 minor:164 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/c433e711922a6551af8477a389f30e4b4d0c73ee2ae134308bc11f81b69f6759/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/662d8345c3f743dad31261a8d9a2699e827b23ef3d7cf5d6150deaca15fa00c8/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/f01246434deacf90078c20ecebb1b7d9fd0f27d5ba7c5c4f0e4be7172b01a639/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/23fe31a758846afc3dc7475816b3a12180f63178ac55474e4bf89019469b4aef/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/e1607ec34d22efe4c9f919c76b7bed1ab4a1278483cf8271c21be939891ac794/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/9f6d2f0e46a04199879f8412e435ab4da8a532a9a14cc015b72bd9d214096ce5/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/11409e1705e450863622e9d5615466bc7a4f92226b04a5ac9b4733e7118df62e/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/eace1f4f40e2e713eaa5ce9d1b6d0635d6f943f3b79e4bb52387e19af9978eb1/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/fe8adfc5b201387f4f6b6d15eed9e3295e3f41a74dab5f31b9a5bdda161a6287/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-219:{mountpoint:/var/lib/containers/storage/overlay/f76e8b9b6581e029b94e395c1f880b4aaa1ba3466c0289ba2da9feb317017ef9/merged major:0 minor:219 fsType:overlay blockSize:0} overlay_0-233:{mountpoint:/var/lib/containers/storage/overlay/806674d372e45cae77185a12e2c2f70f6fe0b5f6c26bce31b026685e0f2b7d88/merged major:0 minor:233 fsType:overlay blockSize:0} overlay_0-235:{mountpoint:/var/lib/containers/storage/overlay/4561c37185107f05ec160aed2028ce2398c8b435002e49233f35164b320661ad/merged major:0 minor:235 fsType:overlay blockSize:0} overlay_0-237:{mountpoint:/var/lib/containers/storage/overlay/cd6bea89684d7bac1f99e20ed46fa85504b9b9778969f792e8d196a851cf9ad8/merged major:0 minor:237 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/820d3efd70a2d00a9407497ab8a38fcaee4cb2677bbf03201f7fea0f09f749a3/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/ac4faaa3150dac3336046864299f87b1884ed9b86b50af29cba4b990c69e85fd/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-267:{mountpoint:/var/lib/containers/storage/overlay/8fa2d83c4f92482c688a3ed187e93aab681ab94bff1e374c323eae2689c4b06d/merged major:0 minor:267 fsType:overlay blockSize:0} overlay_0-270:{mountpoint:/var/lib/containers/storage/overlay/129c79cde1950a49fac3aadbd8b0d1481655f28e1d2ae582045c8b2b21cc0554/merged major:0 minor:270 fsType:overlay blockSize:0} overlay_0-272:{mountpoint:/var/lib/containers/storage/overlay/6465afe64f62c15d65581bc9cc017380b317fe2cf0eb930af36c46db31515f29/merged major:0 minor:272 fsType:overlay blockSize:0} overlay_0-273:{mountpoint:/var/lib/containers/storage/overlay/86598c957b637b0d08dc289714e90615799636a0b4a51f0a0b58a56af5443279/merged major:0 minor:273 fsType:overlay blockSize:0} overlay_0-284:{mountpoint:/var/lib/containers/storage/overlay/c2f4e6b5e532ac3e3fc0629129ee122232fa3e3c316e45b26457620bed370b3b/merged major:0 minor:284 fsType:overlay blockSize:0} overlay_0-291:{mountpoint:/var/lib/containers/storage/overlay/2575b229d05550b58595d2ea0872fbc4d9c89c8ddab95a687f5de883ee165661/merged major:0 minor:291 fsType:overlay blockSize:0} overlay_0-294:{mountpoint:/var/lib/containers/storage/overlay/4b6c104c6ed7875a3528f1589439dc581e7a77f54e39a28de40a30ba25cbf4d8/merged major:0 minor:294 fsType:overlay blockSize:0} overlay_0-305:{mountpoint:/var/lib/containers/storage/overlay/58d11ba66562dbf7432768da654ad3bc0a6a7c6013c2b58236eb158a675632bc/merged major:0 minor:305 fsType:overlay blockSize:0} overlay_0-310:{mountpoint:/var/lib/containers/storage/overlay/9aa177a38a40d3ea164dedfbc90c392e48be3080fdbec6e82dc7261137e5e619/merged major:0 minor:310 fsType:overlay blockSize:0} overlay_0-316:{mountpoint:/var/lib/containers/storage/overlay/fe25f2d1b84ff646e6413c2d30ef6cf38bb54cc3f0e257609438190b4b7037cc/merged major:0 minor:316 fsType:overlay blockSize:0} overlay_0-333:{mountpoint:/var/lib/containers/storage/overlay/5c1946fdfec86412058ed8242bbc9a7461b70e8446dfb843b934acecbaf015ac/merged major:0 minor:333 fsType:overlay blockSize:0} overlay_0-335:{mountpoint:/var/lib/containers/storage/overlay/1fb04ddde55757d83b7695320778d811467cb002b62845cd5d756c99ebb1384d/merged major:0 minor:335 fsType:overlay blockSize:0} overlay_0-340:{mountpoint:/var/lib/containers/storage/overlay/eed0eda4fda2544e0424c0a6d6a190b85bf0bad0e4855d5a317f82fa017f9bd6/merged major:0 minor:340 fsType:overlay blockSize:0} overlay_0-345:{mountpoint:/var/lib/containers/storage/overlay/5687d8e7b776a4e39471d6c689423981eae3786cc3edbbe3cde04040cb1fb604/merged major:0 minor:345 fsType:overlay blockSize:0} overlay_0-359:{mountpoint:/var/lib/containers/storage/overlay/f61f6b98574f607eb7b0ba6d0195bcb78a021fbe16366bc5b5b723949f5c838c/merged major:0 minor:359 fsType:overlay blockSize:0} overlay_0-361:{mountpoint:/var/lib/containers/storage/overlay/b807a45bbeed9cc5b90dad920dc6a49aa8cbfdad4e8f957f5fadfd60b77dff19/merged major:0 minor:361 fsType:overlay blockSize:0} overlay_0-369:{mountpoint:/var/lib/containers/storage/overlay/5fd4a514663ca6b9c2f1db028a6bd79996e8a218bae819eb6b680c0bf0f90815/merged major:0 minor:369 fsType:overlay blockSize:0} overlay_0-373:{mountpoint:/var/lib/containers/storage/overlay/5ba5da0537cded15fdc36eb0126db1f8b3e8541e8908494bba124fee82492289/merged major:0 minor:373 fsType:overlay blockSize:0} overlay_0-376:{mountpoint:/var/lib/containers/storage/overlay/a44a55814c2dcae0e7a468c2b3b0d12cc6120fd95bccceb450c619fb21ceb2b6/merged major:0 minor:376 fsType:overlay blockSize:0} overlay_0-381:{mountpoint:/var/lib/containers/storage/overlay/3ee834c8f318e79dfde2942effc4cdf51b4fea79945c95186ead674645f9f69b/merged major:0 minor:381 fsType:overlay blockSize:0} overlay_0-387:{mountpoint:/var/lib/containers/storage/overlay/5ee3a3b3355fc908e3a6573a72c9982f969009adebf524d0034a0fbf17df7375/merged major:0 minor:387 fsType:overlay blockSize:0} overlay_0-392:{mountpoint:/var/lib/containers/storage/overlay/2a8d092576c7e9a704e4b42f1b181b6c0037b8d06b19655fa8d5873c16226427/merged major:0 minor:392 fsType:overlay blockSize:0} overlay_0-394:{mountpoint:/var/lib/containers/storage/overlay/531e31ca24feb3be35c106f26e61541d16a7d6625fe45c2808207ab49f31b2eb/merged major:0 minor:394 fsType:overlay blockSize:0} overlay_0-399:{mountpoint:/var/lib/containers/storage/overlay/a8dc29f0de711531ef66a0dbec0a47bf4060c51ea00f09d0e9cdaa22e81b61c7/merged major:0 minor:399 fsType:overlay blockSize:0} overlay_0-404:{mountpoint:/var/lib/containers/storage/overlay/8e35b07978ed9e83c05e5f42a64ee93e638b3a8f0af4a6ae4482ee26b1e3a3d3/merged major:0 minor:404 fsType:overlay blockSize:0} overlay_0-409:{mountpoint:/var/lib/containers/storage/overlay/3bde38ce4694af4162c724c6bda25efcba511e572c26086bb5b00ec4e41eb1cd/merged major:0 minor:409 fsType:overlay blockSize:0} overlay_0-414:{mountpoint:/var/lib/containers/storage/overlay/ead4f3dface87f77243d37b6dd0a213d52b5d7ae7db0ff0c015ee5a7a9d7c9f5/merged major:0 minor:414 fsType:overlay blockSize:0} overlay_0-416:{mountpoint:/var/lib/containers/storage/overlay/7d547f0fde342a31fefca308614843a9d0ada61070a4af054a1312395699ce63/merged major:0 minor:416 fsType:overlay blockSize:0} overlay_0-424:{mountpoint:/var/lib/containers/storage/overlay/6ba7cbe815437b6c48268b42ebe9ef36ccd4e2eb6f7f13de3a6bedf668ac8c10/merged major:0 minor:424 fsType:overlay blockSize:0} overlay_0-429:{mountpoint:/var/lib/containers/storage/overlay/55fa718de1f7f4ddb667cff1ad24f12e54685729f0b27c9236c7f9b36155ff6e/merged major:0 minor:429 fsType:overlay blockSize:0} overlay_0-458:{mountpoint:/var/lib/containers/storage/overlay/a5e813f4b3cb7c7ecabf4d90a92b019ee762b17e2bf9812d8d3f8cff58c4ba96/merged major:0 minor:458 fsType:overlay blockSize:0} overlay_0-460:{mountpoint:/var/lib/containers/storage/overlay/63a9182f2e41509ca743721580fcd032d098efb58a8c1bb87a0fda759bbd2d14/merged major:0 minor:460 fsType:overlay blockSize:0} overlay_0-462:{mountpoint:/var/lib/containers/storage/overlay/36a676c7fae4e8febe05fca96b1bd94e52111e70209c8d62222f539b84bba0b9/merged major:0 minor:462 fsType:overlay blockSize:0} overlay_0-468:{mountpoint:/var/lib/containers/storage/overlay/00ffc17565eb19895aa955b933247e9e1c6b0cb7bede732f274dbb0f793db0d7/merged major:0 minor:468 fsType:overlay blockSize:0} overlay_0-470:{mountpoint:/var/lib/containers/storage/overlay/73003bb9980f19584b50ad1d9cdba0b06c7aca891cdc21821134c835f75bbbad/merged major:0 minor:470 fsType:overlay blockSize:0} overlay_0-476:{mountpoint:/var/lib/containers/storage/overlay/e02bfcdfdb8a997a323c8b8c9c2468628e94ae05640728e18bf86f7f3bdf633b/merged major:0 minor:476 fsType:overlay blockSize:0} overlay_0-481:{mountpoint:/var/lib/containers/storage/overlay/5ae0135fd3ae9c91c86b1334183ebab7b0fb94bd882477e9e9d30c2179e5eacf/merged major:0 minor:481 fsType:overlay blockSize:0} overlay_0-486:{mountpoint:/var/lib/containers/storage/overlay/4012529bfa21ee23f92e2c8e86bb0771cd6d7d59e0e44208a90e6411b2fbad23/merged major:0 minor:486 fsType:overlay blockSize:0} overlay_0-491:{mountpoint:/var/lib/containers/storage/overlay/1c70df62dde983c0854eaaaa6e3036259b9de14f9fa82fa7899230e308aadf95/merged major:0 minor:491 fsType:overlay blockSize:0} overlay_0-508:{mountpoint:/var/lib/containers/storage/overlay/5afca669baa848f28a934a14f217a83dfd9ab7a06c6e3754fa7463273fc9ef2e/merged major:0 minor:508 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/bc023411030152672ee1a3f44e1eadb1a914759e5bd30bf9addc3034856702a1/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-510:{mountpoint:/var/lib/containers/storage/overlay/51328fa031732ecb364cca3bf7f77bb4acd1926cdb05c200e38ab570bb9113a9/merged major:0 minor:510 fsType:overlay blockSize:0} overlay_0-512:{mountpoint:/var/lib/containers/storage/overlay/aac212b1ac8a2eccf40bd9ce8dec56ee421c3e8a1887a1b77f821c027e121bc6/merged major:0 minor:512 fsType:overlay blockSize:0} overlay_0-514:{mountpoint:/var/lib/containers/storage/overlay/3d4ac211323e3a5200d404bdba066ec72384f2f1ef58186762319f21bdb938d2/merged major:0 minor:514 fsType:overlay blockSize:0} overlay_0-516:{mountpoint:/var/lib/containers/storage/overlay/495912b05c4854fc3d0ace947f65156f9bba74fd635ad91c2dd0e3733011449e/merged major:0 minor:516 fsType:overlay blockSize:0} overlay_0-522:{mountpoint:/var/lib/containers/storage/overlay/5a5d0b49e6274051465e396031aff8a385547f9ea9874c4d11daeeda3dc38273/merged major:0 minor:522 fsType:overlay blockSize:0} overlay_0-527:{mountpoint:/var/lib/containers/storage/overlay/f9c6a36950c0372896b8f3c4d14bb3d2517686a62d495bc6b693309a4484fd2b/merged major:0 minor:527 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/8f1fe93551120fa087ab6d79dd623d1a066884317d4ea4ce8eb1fcf5c4981b1a/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/749c286b22dcc2fb317aaf716bd4eb752da2407ad3982deecc1572d6222722cd/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/75774488a0e87efd2022170357d168a9147d6daf79e53aaac7bef0db0da3e664/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/0681676da0ff13511a7168b7aac1c7cff3d8ed058c06fcdefcf0fd1de24da30c/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/1843866377fee342886b69dca1e9f827e974c9643a5ca9cabfef3e1621a30f57/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/fa9d2a6d48655da72006ada08e917a3c4fd8fc17cb59436b31a9f661daedfb15/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/25f1c4f4c47322b22b9b7ef25f1d680f88d94b251d7a16feb56391c13edc1b25/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/fb342a5f81bfe6fd7ad82a08ce835453797379ae43bec9086db36a0127d8231b/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/0b4678ea53e0a9a9a1f070b3cca63e5a6c0b5fe18011eef5428cf1b6c839f3dc/merged major:0 minor:96 fsType:overlay blockSize:0}]
Oct 30 19:21:28.223828 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.223403    5907 manager.go:217] Machine: {Timestamp:2026-10-30 19:21:28.222843554 +0000 UTC m=+0.065775484 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100166 MemoryCapacity:32812015616 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec268f2c3424d8ca562362417016b90e SystemUUID:ec268f2c-3424-d8ca-5623-62417016b90e BootID:c43b134f-0af3-4ca4-adbe-6b35b4b990c9 Filesystems:[{Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:overlay_0-142 DeviceMajor:0 DeviceMinor:142 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c59e8b44-50b7-4a50-af6a-1bb9a7547a56/volumes/kubernetes.io~projected/kube-api-access-t5c8j DeviceMajor:0 DeviceMinor:208 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f1622f2e-319b-4259-b314-a4178b7875f0/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:256 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:297 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-294 DeviceMajor:0 DeviceMinor:294 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-414 DeviceMajor:0 DeviceMinor:414 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1ceff922-50ab-4e0f-b113-03719eb40bc9/volumes/kubernetes.io~projected/kube-api-access-skgdt DeviceMajor:0 DeviceMinor:60 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric DeviceMajor:0 DeviceMinor:299 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-394 DeviceMajor:0 DeviceMinor:394 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-161 DeviceMajor:0 DeviceMinor:161 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-272 DeviceMajor:0 DeviceMinor:272 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3b05813e-ce3d-4e71-a64e-7417997c2988/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics DeviceMajor:0 DeviceMinor:327 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volume-subpaths/web-config/init-config-reloader/6 DeviceMajor:0 DeviceMinor:297 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-373 DeviceMajor:0 DeviceMinor:373 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-404 DeviceMajor:0 DeviceMinor:404 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:292 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fe16d935efdb60f47b80f91c0cd9440e9da8c1aa652bdf06a543d6ccec9b3708/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ba4ee546-c9c1-4573-b11f-6ba0621972d1/volumes/kubernetes.io~projected/kube-api-access-rwf55 DeviceMajor:0 DeviceMinor:59 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a4ed12fa-0026-4bf8-a99c-83b04caac774/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:243 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f9f6d62c512fcf71a180244d70043254b9c542a889ca7b3feecd9db2ef1a70d2/userdata/shm DeviceMajor:0 DeviceMinor:263 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volume-subpaths/cluster-tls-config/alertmanager/10 DeviceMajor:0 DeviceMinor:301 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file DeviceMajor:0 DeviceMinor:453 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/a8635205-1f61-4a17-bc1f-7876c5536f9c/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:57 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6ac107b4bbb8ab01e09738c400166c19b041992f42a2ea5a87e53022c6317bd2/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/454d4e22040463e0ef128cd01598ac2291d8257d485ab2493387e223af92cf71/userdata/shm DeviceMajor:0 DeviceMinor:367 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-416 DeviceMajor:0 DeviceMinor:416 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-458 DeviceMajor:0 DeviceMinor:458 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cd444ba1-edf1-49aa-a0cf-b863c93e60be/volumes/kubernetes.io~projected/kube-api-access-c2xwk DeviceMajor:0 DeviceMinor:503 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2c840c6b-6992-4f29-9023-a0ad99fdc543/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:58 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-219 DeviceMajor:0 DeviceMinor:219 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-392 DeviceMajor:0 DeviceMinor:392 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls DeviceMajor:0 DeviceMinor:450 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-522 DeviceMajor:0 DeviceMinor:522 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f1622f2e-319b-4259-b314-a4178b7875f0/volumes/kubernetes.io~projected/kube-api-access-wmftk DeviceMajor:0 DeviceMinor:259 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/23ec668e-b2fb-471f-a7c7-13095a3dcf7f/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:56 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8e6694fb5f64111134a16b0c70a9d467732ebe834c694bb55f0c657b98a0c1fb/userdata/shm DeviceMajor:0 DeviceMinor:81 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c37476d3d2fccb3f1018e0edaaeacc5cacdba55633f63c064597d568e0345fa6/userdata/shm DeviceMajor:0 DeviceMinor:206 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3e2295a83987368051b0ecf5c6b265d334ed32ee0f27fd4214a29d995ad7fd9f/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-235 DeviceMajor:0 DeviceMinor:235 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b41606bf-e8ca-4fc6-8805-b7461187d65e/volumes/kubernetes.io~projected/kube-api-access-bln4w DeviceMajor:0 DeviceMinor:64 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56/volumes/kubernetes.io~projected/kube-api-access-vv49j DeviceMajor:0 DeviceMinor:436 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~secret/secret-kube-rbac-proxy DeviceMajor:0 DeviceMinor:448 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-491 DeviceMajor:0 DeviceMinor:491 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/57769216-dd32-4edc-8e73-b4873caa041d/volumes/kubernetes.io~projected/kube-api-access-4rvqr DeviceMajor:0 DeviceMinor:67 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/399dc2728fd6bfe75e8e8f5ea5a208feb42abca5755c9ec2323bdc27e2afc8a5/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d007f9d11dde4a56571ed3b2441f4f3d884c8bfeb549eecd1a362a77805a1420/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/3b05813e-ce3d-4e71-a64e-7417997c2988/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules DeviceMajor:0 DeviceMinor:328 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volume-subpaths/web-config/alertmanager/9 DeviceMajor:0 DeviceMinor:297 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-510 DeviceMajor:0 DeviceMinor:510 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3b05813e-ce3d-4e71-a64e-7417997c2988/volumes/kubernetes.io~secret/secret-thanos-querier-tls DeviceMajor:0 DeviceMinor:325 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/3b05813e-ce3d-4e71-a64e-7417997c2988/volumes/kubernetes.io~projected/kube-api-access-jqrkv DeviceMajor:0 DeviceMinor:330 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/41ff5ee2-70a2-4baf-af4e-00e6575c2e99/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:500 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-527 DeviceMajor:0 DeviceMinor:527 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a8635205-1f61-4a17-bc1f-7876c5536f9c/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:55 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c97b572cfac5edb55782195f4e1400e2bbec702087499bfdac85303de5ba47a5/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ebe32d9b522ba42b347de33cfc25c86867ad907ece79d0455ede7a3820a1c32d/userdata/shm DeviceMajor:0 DeviceMinor:265 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-273 DeviceMajor:0 DeviceMinor:273 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volume-subpaths/web-config/prometheus/15 DeviceMajor:0 DeviceMinor:446 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e6f51ee7798f1661bfe55e1c8b7ce3927025c79b69d52572c9c29603d43dcf53/userdata/shm DeviceMajor:0 DeviceMinor:501 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/a8635205-1f61-4a17-bc1f-7876c5536f9c/volumes/kubernetes.io~projected/kube-api-access-txjsq DeviceMajor:0 DeviceMinor:61 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/23ec668e-b2fb-471f-a7c7-13095a3dcf7f/volumes/kubernetes.io~projected/kube-api-access-5dls7 DeviceMajor:0 DeviceMinor:66 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/23ec668e-b2fb-471f-a7c7-13095a3dcf7f/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/9a1d2cec-f539-4106-a185-3134a3df747c/volumes/kubernetes.io~projected/kube-api-access-8k9b2 DeviceMajor:0 DeviceMinor:202 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9f818a927ce3a7d5f3fe85edc4285eb3f0932cc7b6bfc950eec0956ed2e70d7f/userdata/shm DeviceMajor:0 DeviceMinor:231 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volumes/kubernetes.io~secret/cluster-tls-config DeviceMajor:0 DeviceMinor:301 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-462 DeviceMajor:0 DeviceMinor:462 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volume-subpaths/web-config/config-reloader/2 DeviceMajor:0 DeviceMinor:446 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-233 DeviceMajor:0 DeviceMinor:233 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-284 DeviceMajor:0 DeviceMinor:284 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:296 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-333 DeviceMajor:0 DeviceMinor:333 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-481 DeviceMajor:0 DeviceMinor:481 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/44b9fc01-6837-46cd-bf01-40cb8fcb90f5/volumes/kubernetes.io~projected/kube-api-access-6klk6 DeviceMajor:0 DeviceMinor:230 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volumes/kubernetes.io~projected/kube-api-access-6vlf5 DeviceMajor:0 DeviceMinor:303 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/bebff447d4392a1b68f1a0d6050ed163c27381784c1b08ee6f917e60a37309f0/userdata/shm DeviceMajor:0 DeviceMinor:438 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-486 DeviceMajor:0 DeviceMinor:486 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-508 DeviceMajor:0 DeviceMinor:508 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-144 DeviceMajor:0 DeviceMinor:144 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9a1d2cec-f539-4106-a185-3134a3df747c/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:197 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/44b9fc01-6837-46cd-bf01-40cb8fcb90f5/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:225 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/1ffad3bc-f14e-4132-b5ee-3ec84d009252/volumes/kubernetes.io~projected/kube-api-access-mdsrm DeviceMajor:0 DeviceMinor:251 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-512 DeviceMajor:0 DeviceMinor:512 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2ca12196-1929-4ece-a076-d36527b95ff1/volumes/kubernetes.io~secret/secret-metrics-server-client-certs DeviceMajor:0 DeviceMinor:353 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/2ca12196-1929-4ece-a076-d36527b95ff1/volumes/kubernetes.io~secret/secret-metrics-server-tls DeviceMajor:0 DeviceMinor:352 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9653fcfafb20e3486e0d7dff2ce47bd26c384f2b0b50f8f065899a174fc3c0b6/userdata/shm DeviceMajor:0 DeviceMinor:355 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-424 DeviceMajor:0 DeviceMinor:424 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:449 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4b70ef66-ed5b-4917-8bca-36757fee37e3/volumes/kubernetes.io~projected/kube-api-access-mvcz2 DeviceMajor:0 DeviceMinor:201 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/990f36d93789ddcfa3d80eae721de26344c8e54c43783b60e92e3128c3a8d2ae/userdata/shm DeviceMajor:0 DeviceMinor:280 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-316 DeviceMajor:0 DeviceMinor:316 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3b05813e-ce3d-4e71-a64e-7417997c2988/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:321 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-359 DeviceMajor:0 DeviceMinor:359 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-376 DeviceMajor:0 DeviceMinor:376 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a7ab6ba877af77a68f6d36be8910e84f896e34840a6913fa1266f5a943f91917/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1ffad3bc-f14e-4132-b5ee-3ec84d009252/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:247 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:300 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volumes/kubernetes.io~secret/secret-alertmanager-main-tls DeviceMajor:0 DeviceMinor:302 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/aa367dd7-da29-424e-9acc-7706f4377494/volumes/kubernetes.io~projected/kube-api-access-s9wwz DeviceMajor:0 DeviceMinor:358 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-381 DeviceMajor:0 DeviceMinor:381 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:447 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c59e8b44-50b7-4a50-af6a-1bb9a7547a56/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:203 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-305 DeviceMajor:0 DeviceMinor:305 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2ca12196-1929-4ece-a076-d36527b95ff1/volumes/kubernetes.io~projected/kube-api-access-slnw6 DeviceMajor:0 DeviceMinor:354 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-399 DeviceMajor:0 DeviceMinor:399 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~secret/secret-metrics-client-certs DeviceMajor:0 DeviceMinor:452 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4b70ef66-ed5b-4917-8bca-36757fee37e3/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:45 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/a4ed12fa-0026-4bf8-a99c-83b04caac774/volumes/kubernetes.io~projected/kube-api-access-rd95d DeviceMajor:0 DeviceMinor:250 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-460 DeviceMajor:0 DeviceMinor:460 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volume-subpaths/web-config/init-config-reloader/2 DeviceMajor:0 DeviceMinor:446 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-516 DeviceMajor:0 DeviceMinor:516 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/41ff5ee2-70a2-4baf-af4e-00e6575c2e99/volumes/kubernetes.io~projected/kube-api-access-rg2rm DeviceMajor:0 DeviceMinor:65 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/f1622f2e-319b-4259-b314-a4178b7875f0/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:257 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-291 DeviceMajor:0 DeviceMinor:291 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-310 DeviceMajor:0 DeviceMinor:310 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-340 DeviceMajor:0 DeviceMinor:340 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls DeviceMajor:0 DeviceMinor:440 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3b05813e-ce3d-4e71-a64e-7417997c2988/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:329 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:430 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406007808 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/44b9fc01-6837-46cd-bf01-40cb8fcb90f5/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:229 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/02f8343e26b330af5d55a4da95611a92cafd596b71d3f7a2b502f1d64f3f2e6d/userdata/shm DeviceMajor:0 DeviceMinor:254 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/3b05813e-ce3d-4e71-a64e-7417997c2988/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy DeviceMajor:0 DeviceMinor:326 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/725d2cce-2a75-426f-8522-d1ddfbabe1b6/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:496 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/90cf8b7e2c8a82be30790ee5f04c3379be42be441410a7ed654100bd85592c48/userdata/shm DeviceMajor:0 DeviceMinor:506 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a4ed12fa-0026-4bf8-a99c-83b04caac774/volumes/kubernetes.io~secret/openshift-state-metrics-tls DeviceMajor:0 DeviceMinor:249 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volumes/kubernetes.io~secret/config-volume DeviceMajor:0 DeviceMinor:298 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-345 DeviceMajor:0 DeviceMinor:345 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2ca12196-1929-4ece-a076-d36527b95ff1/volumes/kubernetes.io~secret/client-ca-bundle DeviceMajor:0 DeviceMinor:347 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:451 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/88bd605a5c6886f91bc53cf97ae5477a38e46ac2fb529ef77ac3fe8cb5b6cdfd/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-164 DeviceMajor:0 DeviceMinor:164 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volume-subpaths/web-config/config-reloader/6 DeviceMajor:0 DeviceMinor:297 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-476 DeviceMajor:0 DeviceMinor:476 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-237 DeviceMajor:0 DeviceMinor:237 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-270 DeviceMajor:0 DeviceMinor:270 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy DeviceMajor:0 DeviceMinor:295 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-468 DeviceMajor:0 DeviceMinor:468 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/79af4843b7a47f58635d1ee037528e4872d865f5a232bb6cbe3589c670b5197d/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2d7437ffa1bf640b1e1e7c6bf743f745578a07f4ca63d213d6bf15dc27281aea/userdata/shm DeviceMajor:0 DeviceMinor:204 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/aa367dd7-da29-424e-9acc-7706f4377494/volumes/kubernetes.io~secret/monitoring-plugin-cert DeviceMajor:0 DeviceMinor:357 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8fe73d8245fc2351fe48fe3f797c40b32b675cb0d0242e738ed1856a41b38627/userdata/shm DeviceMajor:0 DeviceMinor:456 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-470 DeviceMajor:0 DeviceMinor:470 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1ffad3bc-f14e-4132-b5ee-3ec84d009252/volumes/kubernetes.io~secret/kube-state-metrics-tls DeviceMajor:0 DeviceMinor:248 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-335 DeviceMajor:0 DeviceMinor:335 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-387 DeviceMajor:0 DeviceMinor:387 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~projected/kube-api-access-pqn8c DeviceMajor:0 DeviceMinor:455 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/1fda3255-2b22-40a0-870b-7b940977e197/volumes/kubernetes.io~projected/kube-api-access-5jdxg DeviceMajor:0 DeviceMinor:62 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1ecb2762907f7b9889572ec183e48157056de637daf0bd21003cc49cb96208ef/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-361 DeviceMajor:0 DeviceMinor:361 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-429 DeviceMajor:0 DeviceMinor:429 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:454 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-267 DeviceMajor:0 DeviceMinor:267 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ab80f0926c3314624b09b3a9706dc0fcd4de7fd2272425a7b5e1cc8744e85053/userdata/shm DeviceMajor:0 DeviceMinor:331 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-369 DeviceMajor:0 DeviceMinor:369 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-409 DeviceMajor:0 DeviceMinor:409 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-514 DeviceMajor:0 DeviceMinor:514 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8161e358-9105-4d99-88c1-98c17db09674/volumes/kubernetes.io~projected/kube-api-access-4hcrs DeviceMajor:0 DeviceMinor:63 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:435 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~secret/config DeviceMajor:0 DeviceMinor:445 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:446 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/60ebcfd6f800063ca62814f9d5037ff17ab8021bb25a20a9fb23019628c36457/userdata/shm DeviceMajor:0 DeviceMinor:504 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:02f8343e26b330a MacAddress:ca:c6:3c:b7:20:21 Speed:10000 Mtu:8901} {Name:2d7437ffa1bf640 MacAddress:1e:5a:0a:b1:79:f1 Speed:10000 Mtu:8901} {Name:3e2295a83987368 MacAddress:e2:a6:45:1b:0a:8d Speed:10000 Mtu:8901} {Name:454d4e22040463e MacAddress:c2:cd:b2:3d:e8:90 Speed:10000 Mtu:8901} {Name:60ebcfd6f800063 MacAddress:f6:ce:20:b6:67:21 Speed:10000 Mtu:8901} {Name:8fe73d8245fc235 MacAddress:b2:4e:bf:e0:1b:91 Speed:10000 Mtu:8901} {Name:90cf8b7e2c8a82b MacAddress:c2:56:d7:c2:39:3b Speed:10000 Mtu:8901} {Name:9653fcfafb20e34 MacAddress:46:29:2d:51:8f:67 Speed:10000 Mtu:8901} {Name:990f36d93789ddc MacAddress:fa:a7:8f:96:61:d2 Speed:10000 Mtu:8901} {Name:9f818a927ce3a7d MacAddress:2e:13:4f:0a:23:d4 Speed:10000 Mtu:8901} {Name:ab80f0926c33146 MacAddress:7a:c7:ae:bb:ac:f5 Speed:10000 Mtu:8901} {Name:bebff447d4392a1 MacAddress:8e:f2:bd:a8:9f:5e Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:d4:cf:fe:c1 Speed:0 Mtu:9001} {Name:br-int MacAddress:c2:c5:74:1a:05:70 Speed:0 Mtu:8901} {Name:c37476d3d2fccb3 MacAddress:ee:46:8d:d9:c4:5a Speed:10000 Mtu:8901} {Name:e6f51ee7798f166 MacAddress:c6:62:96:6a:8c:1a Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:d4:cf:fe:c1 Speed:0 Mtu:9001} {Name:f9f6d62c512fcf7 MacAddress:ee:f6:e3:19:22:c9 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:36:3c:db:e2:d4:a3 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:86:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:6a:8a:67:d1:90:ba Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812015616 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 19:21:28.223828 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.223829    5907 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 19:21:28.224218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.223903    5907 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 19:21:28.224252 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.224229    5907 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 19:21:28.224383 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.224252    5907 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-152.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 19:21:28.224430 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.224385    5907 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 19:21:28.224430 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.224392    5907 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 19:21:28.224430 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.224402    5907 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 19:21:28.224430 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.224421    5907 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 19:21:28.224539 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.224511    5907 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 19:21:28.224711 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.224701    5907 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 19:21:28.224825 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.224816    5907 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 19:21:28.224935 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.224853    5907 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 19:21:28.224935 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.224895    5907 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 19:21:28.224935 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.224906    5907 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 19:21:28.224935 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.224916    5907 kubelet.go:405] "Adding apiserver pod source"
Oct 30 19:21:28.224935 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.224924    5907 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 19:21:28.225724 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.225711    5907 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 19:21:28.225724 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.225723    5907 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 19:21:28.226290 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.226270    5907 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 19:21:28.230442 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.230425    5907 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 19:21:28.230442 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.230439    5907 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 19:21:28.230559 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.230550    5907 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 19:21:28.230751 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.230740    5907 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 19:21:28.230781 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.230769    5907 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 19:21:28.230889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.230878    5907 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 19:21:28.230927 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.230900    5907 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 19:21:28.230927 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.230907    5907 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 19:21:28.230927 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.230921    5907 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 19:21:28.230927 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.230927    5907 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 19:21:28.231035 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.230933    5907 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 19:21:28.231035 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.230939    5907 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 19:21:28.231035 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.230944    5907 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 19:21:28.231035 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.230952    5907 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 19:21:28.231035 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.230957    5907 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 19:21:28.231035 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.230970    5907 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 19:21:28.231035 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.230980    5907 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 19:21:28.231035 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.231007    5907 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 19:21:28.231035 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.231016    5907 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 19:21:28.234508 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.234495    5907 server.go:1263] "Started kubelet"
Oct 30 19:21:28.234617 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.234572    5907 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 19:21:28.234749 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.234599    5907 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 19:21:28.234749 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.234675    5907 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 19:21:28.235034 ip-10-0-132-152 systemd[1]: Started Kubernetes Kubelet.
Oct 30 19:21:28.235223 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.235129    5907 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 19:21:28.236995 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.236557    5907 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 19:21:28.240460 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.240439    5907 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 19:21:28.240666 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.240644    5907 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 19:21:28.240723 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.240667    5907 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 19:21:28.241368 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:21:28.241343    5907 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-152.ec2.internal\" not found"
Oct 30 19:21:28.241471 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.240653    5907 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 19:21:28.241471 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.241449    5907 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 19:21:28.241710 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.241685    5907 factory.go:55] Registering systemd factory
Oct 30 19:21:28.241793 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.241716    5907 factory.go:223] Registration of the systemd container factory successfully
Oct 30 19:21:28.241793 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.241674    5907 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 19:15:19 +0000 UTC" deadline="2028-08-09 02:06:59.606855248 +0000 UTC"
Oct 30 19:21:28.241793 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.241763    5907 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15558h45m31.365097173s"
Oct 30 19:21:28.242309 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.242290    5907 factory.go:153] Registering CRI-O factory
Oct 30 19:21:28.242309 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.242310    5907 factory.go:223] Registration of the crio container factory successfully
Oct 30 19:21:28.242463 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.242379    5907 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 19:21:28.242463 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.242401    5907 factory.go:103] Registering Raw factory
Oct 30 19:21:28.242463 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.242416    5907 manager.go:1199] Started watching for new ooms in manager
Oct 30 19:21:28.242746 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.242731    5907 manager.go:319] Starting recovery of all containers
Oct 30 19:21:28.245939 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.245892    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a4ed12fa-0026-4bf8-a99c-83b04caac774" volumeName="kubernetes.io/configmap/a4ed12fa-0026-4bf8-a99c-83b04caac774-metrics-client-ca" seLinuxMountContext=""
Oct 30 19:21:28.246046 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.245943    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a8635205-1f61-4a17-bc1f-7876c5536f9c" volumeName="kubernetes.io/empty-dir/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-tuned" seLinuxMountContext=""
Oct 30 19:21:28.246046 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.245960    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" volumeName="kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-prometheus-k8s-tls" seLinuxMountContext=""
Oct 30 19:21:28.246046 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.245975    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56" volumeName="kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 19:21:28.246046 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.245990    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9a1d2cec-f539-4106-a185-3134a3df747c" volumeName="kubernetes.io/projected/9a1d2cec-f539-4106-a185-3134a3df747c-kube-api-access-8k9b2" seLinuxMountContext=""
Oct 30 19:21:28.246046 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246004    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ba4ee546-c9c1-4573-b11f-6ba0621972d1" volumeName="kubernetes.io/projected/ba4ee546-c9c1-4573-b11f-6ba0621972d1-kube-api-access-rwf55" seLinuxMountContext=""
Oct 30 19:21:28.246046 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246017    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" volumeName="kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-rulefiles-0" seLinuxMountContext=""
Oct 30 19:21:28.246046 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246029    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f1622f2e-319b-4259-b314-a4178b7875f0" volumeName="kubernetes.io/configmap/f1622f2e-319b-4259-b314-a4178b7875f0-metrics-client-ca" seLinuxMountContext=""
Oct 30 19:21:28.246046 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246046    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1ffad3bc-f14e-4132-b5ee-3ec84d009252" volumeName="kubernetes.io/empty-dir/1ffad3bc-f14e-4132-b5ee-3ec84d009252-volume-directive-shadow" seLinuxMountContext=""
Oct 30 19:21:28.246427 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246062    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="40121914-4632-47b7-bc42-8299b3d6d6bd" volumeName="kubernetes.io/empty-dir/40121914-4632-47b7-bc42-8299b3d6d6bd-config-out" seLinuxMountContext=""
Oct 30 19:21:28.246427 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246077    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ba4ee546-c9c1-4573-b11f-6ba0621972d1" volumeName="kubernetes.io/configmap/ba4ee546-c9c1-4573-b11f-6ba0621972d1-serviceca" seLinuxMountContext=""
Oct 30 19:21:28.246427 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246091    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f1622f2e-319b-4259-b314-a4178b7875f0" volumeName="kubernetes.io/secret/f1622f2e-319b-4259-b314-a4178b7875f0-node-exporter-tls" seLinuxMountContext=""
Oct 30 19:21:28.246427 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246103    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" volumeName="kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-rulefiles-1" seLinuxMountContext=""
Oct 30 19:21:28.246427 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246136    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" volumeName="kubernetes.io/empty-dir/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-db" seLinuxMountContext=""
Oct 30 19:21:28.246427 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246153    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3b05813e-ce3d-4e71-a64e-7417997c2988" volumeName="kubernetes.io/configmap/3b05813e-ce3d-4e71-a64e-7417997c2988-metrics-client-ca" seLinuxMountContext=""
Oct 30 19:21:28.246427 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246166    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a4ed12fa-0026-4bf8-a99c-83b04caac774" volumeName="kubernetes.io/secret/a4ed12fa-0026-4bf8-a99c-83b04caac774-openshift-state-metrics-tls" seLinuxMountContext=""
Oct 30 19:21:28.246427 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246179    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c59e8b44-50b7-4a50-af6a-1bb9a7547a56" volumeName="kubernetes.io/secret/c59e8b44-50b7-4a50-af6a-1bb9a7547a56-cert" seLinuxMountContext=""
Oct 30 19:21:28.246427 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246210    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1ffad3bc-f14e-4132-b5ee-3ec84d009252" volumeName="kubernetes.io/configmap/1ffad3bc-f14e-4132-b5ee-3ec84d009252-kube-state-metrics-custom-resource-state-configmap" seLinuxMountContext=""
Oct 30 19:21:28.246427 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246237    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ca12196-1929-4ece-a076-d36527b95ff1" volumeName="kubernetes.io/secret/2ca12196-1929-4ece-a076-d36527b95ff1-secret-metrics-server-tls" seLinuxMountContext=""
Oct 30 19:21:28.246427 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246250    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3b05813e-ce3d-4e71-a64e-7417997c2988" volumeName="kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-grpc-tls" seLinuxMountContext=""
Oct 30 19:21:28.246427 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246263    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="44b9fc01-6837-46cd-bf01-40cb8fcb90f5" volumeName="kubernetes.io/configmap/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-service-ca" seLinuxMountContext=""
Oct 30 19:21:28.246427 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246276    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="57769216-dd32-4edc-8e73-b4873caa041d" volumeName="kubernetes.io/configmap/57769216-dd32-4edc-8e73-b4873caa041d-iptables-alerter-script" seLinuxMountContext=""
Oct 30 19:21:28.246427 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246289    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1fda3255-2b22-40a0-870b-7b940977e197" volumeName="kubernetes.io/configmap/1fda3255-2b22-40a0-870b-7b940977e197-cni-binary-copy" seLinuxMountContext=""
Oct 30 19:21:28.246427 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246303    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1ffad3bc-f14e-4132-b5ee-3ec84d009252" volumeName="kubernetes.io/secret/1ffad3bc-f14e-4132-b5ee-3ec84d009252-kube-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 19:21:28.246427 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246316    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cd444ba1-edf1-49aa-a0cf-b863c93e60be" volumeName="kubernetes.io/projected/cd444ba1-edf1-49aa-a0cf-b863c93e60be-kube-api-access-c2xwk" seLinuxMountContext=""
Oct 30 19:21:28.246427 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246328    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="23ec668e-b2fb-471f-a7c7-13095a3dcf7f" volumeName="kubernetes.io/configmap/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-env-overrides" seLinuxMountContext=""
Oct 30 19:21:28.246427 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246341    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4b70ef66-ed5b-4917-8bca-36757fee37e3" volumeName="kubernetes.io/configmap/4b70ef66-ed5b-4917-8bca-36757fee37e3-config-volume" seLinuxMountContext=""
Oct 30 19:21:28.246427 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246354    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="23ec668e-b2fb-471f-a7c7-13095a3dcf7f" volumeName="kubernetes.io/configmap/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-ovnkube-config" seLinuxMountContext=""
Oct 30 19:21:28.246427 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246371    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="44b9fc01-6837-46cd-bf01-40cb8fcb90f5" volumeName="kubernetes.io/configmap/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-oauth-serving-cert" seLinuxMountContext=""
Oct 30 19:21:28.246427 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246390    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56" volumeName="kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-console-config" seLinuxMountContext=""
Oct 30 19:21:28.246427 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246404    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56" volumeName="kubernetes.io/secret/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-console-oauth-config" seLinuxMountContext=""
Oct 30 19:21:28.246427 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246416    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" volumeName="kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-metrics-client-certs" seLinuxMountContext=""
Oct 30 19:21:28.246427 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246430    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="40121914-4632-47b7-bc42-8299b3d6d6bd" volumeName="kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-kube-rbac-proxy-metric" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246445    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1ceff922-50ab-4e0f-b113-03719eb40bc9" volumeName="kubernetes.io/projected/1ceff922-50ab-4e0f-b113-03719eb40bc9-kube-api-access-skgdt" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246458    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4b70ef66-ed5b-4917-8bca-36757fee37e3" volumeName="kubernetes.io/secret/4b70ef66-ed5b-4917-8bca-36757fee37e3-metrics-tls" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246473    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="725d2cce-2a75-426f-8522-d1ddfbabe1b6" volumeName="kubernetes.io/secret/725d2cce-2a75-426f-8522-d1ddfbabe1b6-original-pull-secret" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246486    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9a1d2cec-f539-4106-a185-3134a3df747c" volumeName="kubernetes.io/secret/9a1d2cec-f539-4106-a185-3134a3df747c-insights-runtime-extractor-tls" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246499    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" volumeName="kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-prometheus-k8s-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246511    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3b05813e-ce3d-4e71-a64e-7417997c2988" volumeName="kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-thanos-querier-kube-rbac-proxy-metrics" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246524    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="40121914-4632-47b7-bc42-8299b3d6d6bd" volumeName="kubernetes.io/empty-dir/40121914-4632-47b7-bc42-8299b3d6d6bd-alertmanager-main-db" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246536    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9a1d2cec-f539-4106-a185-3134a3df747c" volumeName="kubernetes.io/configmap/9a1d2cec-f539-4106-a185-3134a3df747c-kube-rbac-proxy-cm" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246551    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1ceff922-50ab-4e0f-b113-03719eb40bc9" volumeName="kubernetes.io/configmap/1ceff922-50ab-4e0f-b113-03719eb40bc9-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246564    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56" volumeName="kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-service-ca" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246578    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a8635205-1f61-4a17-bc1f-7876c5536f9c" volumeName="kubernetes.io/projected/a8635205-1f61-4a17-bc1f-7876c5536f9c-kube-api-access-txjsq" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246592    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" volumeName="kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-web-config" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246604    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2c840c6b-6992-4f29-9023-a0ad99fdc543" volumeName="kubernetes.io/configmap/2c840c6b-6992-4f29-9023-a0ad99fdc543-konnectivity-ca" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246617    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3b05813e-ce3d-4e71-a64e-7417997c2988" volumeName="kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-thanos-querier-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246631    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="44b9fc01-6837-46cd-bf01-40cb8fcb90f5" volumeName="kubernetes.io/secret/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-console-serving-cert" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246643    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="40121914-4632-47b7-bc42-8299b3d6d6bd" volumeName="kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-main-tls" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246657    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b41606bf-e8ca-4fc6-8805-b7461187d65e" volumeName="kubernetes.io/empty-dir/b41606bf-e8ca-4fc6-8805-b7461187d65e-tmp-dir" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246670    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="40121914-4632-47b7-bc42-8299b3d6d6bd" volumeName="kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-web-config" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246683    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="41ff5ee2-70a2-4baf-af4e-00e6575c2e99" volumeName="kubernetes.io/projected/41ff5ee2-70a2-4baf-af4e-00e6575c2e99-kube-api-access-rg2rm" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246696    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1fda3255-2b22-40a0-870b-7b940977e197" volumeName="kubernetes.io/configmap/1fda3255-2b22-40a0-870b-7b940977e197-multus-daemon-config" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246743    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="40121914-4632-47b7-bc42-8299b3d6d6bd" volumeName="kubernetes.io/projected/40121914-4632-47b7-bc42-8299b3d6d6bd-kube-api-access-6vlf5" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246759    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="41ff5ee2-70a2-4baf-af4e-00e6575c2e99" volumeName="kubernetes.io/secret/41ff5ee2-70a2-4baf-af4e-00e6575c2e99-metrics-certs" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246771    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c59e8b44-50b7-4a50-af6a-1bb9a7547a56" volumeName="kubernetes.io/projected/c59e8b44-50b7-4a50-af6a-1bb9a7547a56-kube-api-access-t5c8j" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246793    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a4ed12fa-0026-4bf8-a99c-83b04caac774" volumeName="kubernetes.io/secret/a4ed12fa-0026-4bf8-a99c-83b04caac774-openshift-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246816    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b41606bf-e8ca-4fc6-8805-b7461187d65e" volumeName="kubernetes.io/projected/b41606bf-e8ca-4fc6-8805-b7461187d65e-kube-api-access-bln4w" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246831    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" volumeName="kubernetes.io/projected/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-kube-api-access-pqn8c" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246855    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" volumeName="kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-grpc-tls" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246886    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f1622f2e-319b-4259-b314-a4178b7875f0" volumeName="kubernetes.io/projected/f1622f2e-319b-4259-b314-a4178b7875f0-kube-api-access-wmftk" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246899    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" volumeName="kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-configmap-kubelet-serving-ca-bundle" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246912    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ca12196-1929-4ece-a076-d36527b95ff1" volumeName="kubernetes.io/configmap/2ca12196-1929-4ece-a076-d36527b95ff1-metrics-server-audit-profiles" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246933    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ca12196-1929-4ece-a076-d36527b95ff1" volumeName="kubernetes.io/empty-dir/2ca12196-1929-4ece-a076-d36527b95ff1-audit-log" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.246964    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3b05813e-ce3d-4e71-a64e-7417997c2988" volumeName="kubernetes.io/projected/3b05813e-ce3d-4e71-a64e-7417997c2988-kube-api-access-jqrkv" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247005    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="40121914-4632-47b7-bc42-8299b3d6d6bd" volumeName="kubernetes.io/projected/40121914-4632-47b7-bc42-8299b3d6d6bd-tls-assets" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247019    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a8635205-1f61-4a17-bc1f-7876c5536f9c" volumeName="kubernetes.io/empty-dir/a8635205-1f61-4a17-bc1f-7876c5536f9c-tmp" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247032    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1fda3255-2b22-40a0-870b-7b940977e197" volumeName="kubernetes.io/projected/1fda3255-2b22-40a0-870b-7b940977e197-kube-api-access-5jdxg" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247079    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ca12196-1929-4ece-a076-d36527b95ff1" volumeName="kubernetes.io/configmap/2ca12196-1929-4ece-a076-d36527b95ff1-configmap-kubelet-serving-ca-bundle" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247102    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="40121914-4632-47b7-bc42-8299b3d6d6bd" volumeName="kubernetes.io/configmap/40121914-4632-47b7-bc42-8299b3d6d6bd-metrics-client-ca" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247115    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" volumeName="kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-prometheus-k8s-thanos-sidecar-tls" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247139    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ca12196-1929-4ece-a076-d36527b95ff1" volumeName="kubernetes.io/secret/2ca12196-1929-4ece-a076-d36527b95ff1-secret-metrics-server-client-certs" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247152    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3b05813e-ce3d-4e71-a64e-7417997c2988" volumeName="kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-thanos-querier-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247195    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="40121914-4632-47b7-bc42-8299b3d6d6bd" volumeName="kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-config-volume" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247210    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="44b9fc01-6837-46cd-bf01-40cb8fcb90f5" volumeName="kubernetes.io/projected/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-kube-api-access-6klk6" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247235    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8161e358-9105-4d99-88c1-98c17db09674" volumeName="kubernetes.io/projected/8161e358-9105-4d99-88c1-98c17db09674-kube-api-access-4hcrs" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247248    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1ceff922-50ab-4e0f-b113-03719eb40bc9" volumeName="kubernetes.io/configmap/1ceff922-50ab-4e0f-b113-03719eb40bc9-cni-binary-copy" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247277    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1ffad3bc-f14e-4132-b5ee-3ec84d009252" volumeName="kubernetes.io/secret/1ffad3bc-f14e-4132-b5ee-3ec84d009252-kube-state-metrics-tls" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247293    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="23ec668e-b2fb-471f-a7c7-13095a3dcf7f" volumeName="kubernetes.io/configmap/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247305    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3b05813e-ce3d-4e71-a64e-7417997c2988" volumeName="kubernetes.io/empty-dir/3b05813e-ce3d-4e71-a64e-7417997c2988-tmp" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247328    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4b70ef66-ed5b-4917-8bca-36757fee37e3" volumeName="kubernetes.io/empty-dir/4b70ef66-ed5b-4917-8bca-36757fee37e3-tmp-dir" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247381    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4b70ef66-ed5b-4917-8bca-36757fee37e3" volumeName="kubernetes.io/projected/4b70ef66-ed5b-4917-8bca-36757fee37e3-kube-api-access-mvcz2" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247395    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" volumeName="kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247420    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" volumeName="kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-config" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247467    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3b05813e-ce3d-4e71-a64e-7417997c2988" volumeName="kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-thanos-querier-kube-rbac-proxy-rules" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247480    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9a1d2cec-f539-4106-a185-3134a3df747c" volumeName="kubernetes.io/empty-dir/9a1d2cec-f539-4106-a185-3134a3df747c-data-volume" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247502    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" volumeName="kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-configmap-serving-certs-ca-bundle" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247515    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" volumeName="kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-rulefiles-2" seLinuxMountContext=""
Oct 30 19:21:28.247510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247527    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" volumeName="kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-thanos-prometheus-http-client-file" seLinuxMountContext=""
Oct 30 19:21:28.249999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247573    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1ceff922-50ab-4e0f-b113-03719eb40bc9" volumeName="kubernetes.io/configmap/1ceff922-50ab-4e0f-b113-03719eb40bc9-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 19:21:28.249999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247589    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="23ec668e-b2fb-471f-a7c7-13095a3dcf7f" volumeName="kubernetes.io/projected/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-kube-api-access-5dls7" seLinuxMountContext=""
Oct 30 19:21:28.249999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247601    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ca12196-1929-4ece-a076-d36527b95ff1" volumeName="kubernetes.io/secret/2ca12196-1929-4ece-a076-d36527b95ff1-client-ca-bundle" seLinuxMountContext=""
Oct 30 19:21:28.249999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247650    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="40121914-4632-47b7-bc42-8299b3d6d6bd" volumeName="kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 19:21:28.249999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247662    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="40121914-4632-47b7-bc42-8299b3d6d6bd" volumeName="kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-cluster-tls-config" seLinuxMountContext=""
Oct 30 19:21:28.249999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247675    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="44b9fc01-6837-46cd-bf01-40cb8fcb90f5" volumeName="kubernetes.io/configmap/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-console-config" seLinuxMountContext=""
Oct 30 19:21:28.249999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247701    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56" volumeName="kubernetes.io/secret/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-console-serving-cert" seLinuxMountContext=""
Oct 30 19:21:28.249999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247726    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aa367dd7-da29-424e-9acc-7706f4377494" volumeName="kubernetes.io/secret/aa367dd7-da29-424e-9acc-7706f4377494-monitoring-plugin-cert" seLinuxMountContext=""
Oct 30 19:21:28.249999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247762    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="40121914-4632-47b7-bc42-8299b3d6d6bd" volumeName="kubernetes.io/configmap/40121914-4632-47b7-bc42-8299b3d6d6bd-alertmanager-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 19:21:28.249999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247786    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a4ed12fa-0026-4bf8-a99c-83b04caac774" volumeName="kubernetes.io/projected/a4ed12fa-0026-4bf8-a99c-83b04caac774-kube-api-access-rd95d" seLinuxMountContext=""
Oct 30 19:21:28.249999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247798    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" volumeName="kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 19:21:28.249999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247824    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1ffad3bc-f14e-4132-b5ee-3ec84d009252" volumeName="kubernetes.io/projected/1ffad3bc-f14e-4132-b5ee-3ec84d009252-kube-api-access-mdsrm" seLinuxMountContext=""
Oct 30 19:21:28.249999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247911    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ca12196-1929-4ece-a076-d36527b95ff1" volumeName="kubernetes.io/projected/2ca12196-1929-4ece-a076-d36527b95ff1-kube-api-access-slnw6" seLinuxMountContext=""
Oct 30 19:21:28.249999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247927    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aa367dd7-da29-424e-9acc-7706f4377494" volumeName="kubernetes.io/projected/aa367dd7-da29-424e-9acc-7706f4377494-kube-api-access-s9wwz" seLinuxMountContext=""
Oct 30 19:21:28.249999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247939    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" volumeName="kubernetes.io/projected/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-tls-assets" seLinuxMountContext=""
Oct 30 19:21:28.249999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247968    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="44b9fc01-6837-46cd-bf01-40cb8fcb90f5" volumeName="kubernetes.io/empty-dir/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-tmp" seLinuxMountContext=""
Oct 30 19:21:28.249999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.247994    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" volumeName="kubernetes.io/empty-dir/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-config-out" seLinuxMountContext=""
Oct 30 19:21:28.249999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.248006    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f1622f2e-319b-4259-b314-a4178b7875f0" volumeName="kubernetes.io/configmap/f1622f2e-319b-4259-b314-a4178b7875f0-node-exporter-accelerators-collector-config" seLinuxMountContext=""
Oct 30 19:21:28.249999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.248062    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f1622f2e-319b-4259-b314-a4178b7875f0" volumeName="kubernetes.io/secret/f1622f2e-319b-4259-b314-a4178b7875f0-node-exporter-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 19:21:28.249999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.248076    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1ffad3bc-f14e-4132-b5ee-3ec84d009252" volumeName="kubernetes.io/configmap/1ffad3bc-f14e-4132-b5ee-3ec84d009252-metrics-client-ca" seLinuxMountContext=""
Oct 30 19:21:28.249999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.248088    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3b05813e-ce3d-4e71-a64e-7417997c2988" volumeName="kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-thanos-querier-tls" seLinuxMountContext=""
Oct 30 19:21:28.249999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.248112    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56" volumeName="kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-oauth-serving-cert" seLinuxMountContext=""
Oct 30 19:21:28.249999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.248125    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56" volumeName="kubernetes.io/empty-dir/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-tmp" seLinuxMountContext=""
Oct 30 19:21:28.249999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.248172    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" volumeName="kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-configmap-metrics-client-ca" seLinuxMountContext=""
Oct 30 19:21:28.249999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.248218    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="40121914-4632-47b7-bc42-8299b3d6d6bd" volumeName="kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 19:21:28.249999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.248231    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="44b9fc01-6837-46cd-bf01-40cb8fcb90f5" volumeName="kubernetes.io/secret/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-console-oauth-config" seLinuxMountContext=""
Oct 30 19:21:28.249999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.248245    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f1622f2e-319b-4259-b314-a4178b7875f0" volumeName="kubernetes.io/empty-dir/f1622f2e-319b-4259-b314-a4178b7875f0-node-exporter-textfile" seLinuxMountContext=""
Oct 30 19:21:28.249999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.248284    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56" volumeName="kubernetes.io/projected/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-kube-api-access-vv49j" seLinuxMountContext=""
Oct 30 19:21:28.249999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.248297    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="23ec668e-b2fb-471f-a7c7-13095a3dcf7f" volumeName="kubernetes.io/secret/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 19:21:28.249999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.248311    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2c840c6b-6992-4f29-9023-a0ad99fdc543" volumeName="kubernetes.io/secret/2c840c6b-6992-4f29-9023-a0ad99fdc543-agent-certs" seLinuxMountContext=""
Oct 30 19:21:28.249999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.248338    5907 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="57769216-dd32-4edc-8e73-b4873caa041d" volumeName="kubernetes.io/projected/57769216-dd32-4edc-8e73-b4873caa041d-kube-api-access-4rvqr" seLinuxMountContext=""
Oct 30 19:21:28.249999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.248350    5907 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 19:21:28.249999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.248357    5907 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 19:21:28.275369 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.275298    5907 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 19:21:28.279668 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.279645    5907 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 19:21:28.279668 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.279673    5907 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 19:21:28.279896 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.279694    5907 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 19:21:28.279896 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:21:28.279736    5907 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 19:21:28.291749 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.291725    5907 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 19:21:28.292889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.292833    5907 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:21:28.304546 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.304520    5907 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:21:28.312687 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.312660    5907 generic.go:356] "Generic (PLEG): container finished" podID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerID="595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed" exitCode=0
Oct 30 19:21:28.318851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.318824    5907 generic.go:356] "Generic (PLEG): container finished" podID="1ceff922-50ab-4e0f-b113-03719eb40bc9" containerID="5150e562ea857cb8be8b0f0c54355d38e28eebb95d2639aeaf7188f0b415a7c5" exitCode=0
Oct 30 19:21:28.318851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.318848    5907 generic.go:356] "Generic (PLEG): container finished" podID="1ceff922-50ab-4e0f-b113-03719eb40bc9" containerID="ec405270d7e6b82f7c8acb8a0d945abc44edbea3d17535efce7c8fe8363be98e" exitCode=0
Oct 30 19:21:28.318851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.318855    5907 generic.go:356] "Generic (PLEG): container finished" podID="1ceff922-50ab-4e0f-b113-03719eb40bc9" containerID="dc3f3e002810b5a0c835fb09a6c51df7ef21a21c130bbdfdf220fcc7fd2b52fe" exitCode=0
Oct 30 19:21:28.318851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.318881    5907 generic.go:356] "Generic (PLEG): container finished" podID="1ceff922-50ab-4e0f-b113-03719eb40bc9" containerID="0c7400064cfbec0cd6dfefe6cf1c8790e09d5c9d8a836e99a283a0304494ae42" exitCode=0
Oct 30 19:21:28.319102 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.318891    5907 generic.go:356] "Generic (PLEG): container finished" podID="1ceff922-50ab-4e0f-b113-03719eb40bc9" containerID="994387be7de962068f59d224b4f232811bb40cbf6fb5037937f314175f648464" exitCode=0
Oct 30 19:21:28.319102 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.318899    5907 generic.go:356] "Generic (PLEG): container finished" podID="1ceff922-50ab-4e0f-b113-03719eb40bc9" containerID="2a29f47271063b29b11594ec378707aeb8b1089c9ea2b9a32b53495dabe7468c" exitCode=0
Oct 30 19:21:28.319209 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.319193    5907 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:21:28.319258 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.319247    5907 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:21:28.330186 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.330160    5907 generic.go:356] "Generic (PLEG): container finished" podID="40121914-4632-47b7-bc42-8299b3d6d6bd" containerID="65ce9926fc3f76d0f605f9b84cd747beac60ac1ea3c9e9b2a10d37644104a416" exitCode=0
Oct 30 19:21:28.332493 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.332468    5907 generic.go:356] "Generic (PLEG): container finished" podID="f1622f2e-319b-4259-b314-a4178b7875f0" containerID="29fa2072d656e4a258caef40f204d7257e886f5786ef6a225be8179331270a14" exitCode=0
Oct 30 19:21:28.352279 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.352210    5907 generic.go:356] "Generic (PLEG): container finished" podID="3412174e80de20d346c1e09c794a54c8" containerID="4babb5839e8b3ac96725738008c894105d2676a536e4897b2e08341aa8ca7cd3" exitCode=0
Oct 30 19:21:28.379805 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:21:28.379774    5907 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Oct 30 19:21:28.380215 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.380199    5907 manager.go:324] Recovery completed
Oct 30 19:21:28.407926 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.407895    5907 cpu_manager.go:226] "Starting" policy="none"
Oct 30 19:21:28.407926 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.407919    5907 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 19:21:28.408117 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.407940    5907 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 19:21:28.408117 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.408094    5907 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 19:21:28.408117 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.408106    5907 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 19:21:28.408253 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.408242    5907 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 19:21:28.408284 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.408253    5907 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 19:21:28.408284 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.408259    5907 policy_none.go:50] "Start"
Oct 30 19:21:28.408284 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.408269    5907 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 19:21:28.408284 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.408279    5907 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 19:21:28.409357 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.409346    5907 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 19:21:28.409389 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.409357    5907 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 19:21:28.409389 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.409363    5907 policy_none.go:44] "Start"
Oct 30 19:21:28.414409 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.414367    5907 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 19:21:28.417173 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.417156    5907 manager.go:341] "Starting Device Plugin manager"
Oct 30 19:21:28.417767 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:21:28.417180    5907 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 19:21:28.417767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.417767    5907 server.go:87] "Starting device plugin registration server"
Oct 30 19:21:28.418066 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.418052    5907 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 19:21:28.418135 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.418064    5907 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 19:21:28.418199 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.418169    5907 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 19:21:28.418307 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.418278    5907 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 19:21:28.418365 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.418316    5907 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 19:21:28.418365 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.418321    5907 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 19:21:28.418460 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.418361    5907 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T19:21:28.418303006Z","UUID":"bed65b7f-a0fd-47c0-b5d9-e2b91be0b7d9","Handler":null,"Name":"","Endpoint":""}
Oct 30 19:21:28.419555 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.419540    5907 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 19:21:28.419679 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.419568    5907 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 19:21:28.420322 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:21:28.420285    5907 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 19:21:28.518413 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.518381    5907 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 19:21:28.520557 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.520523    5907 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-152.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 19:21:28.520630 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.520564    5907 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-152.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 19:21:28.520630 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.520576    5907 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-152.ec2.internal" event="NodeHasSufficientPID"
Oct 30 19:21:28.520718 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.520646    5907 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-132-152.ec2.internal"
Oct 30 19:21:28.557210 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.557186    5907 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-132-152.ec2.internal"
Oct 30 19:21:28.557334 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.557272    5907 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-132-152.ec2.internal"
Oct 30 19:21:28.580433 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.580381    5907 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-132-152.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal"]
Oct 30 19:21:28.581115 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.581098    5907 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-152.ec2.internal"
Oct 30 19:21:28.581115 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.581093    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-152.ec2.internal" event={"ID":"e58ff4b1131d98fcba47fc4e4f2a3553","Type":"ContainerStarted","Data":"1b6d05a757c1cc5257fcad2ae658ea4291fcfba280e72e9942db24513330ae12"}
Oct 30 19:21:28.581240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.581128    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-152.ec2.internal" event={"ID":"e58ff4b1131d98fcba47fc4e4f2a3553","Type":"ContainerStarted","Data":"6ac107b4bbb8ab01e09738c400166c19b041992f42a2ea5a87e53022c6317bd2"}
Oct 30 19:21:28.581240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.581193    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal" event={"ID":"3412174e80de20d346c1e09c794a54c8","Type":"ContainerStarted","Data":"58206b1f6bf6af344b08f85b7cd912b8f0268072a20ccb6fc5161b9570bc827c"}
Oct 30 19:21:28.581240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.581204    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal" event={"ID":"3412174e80de20d346c1e09c794a54c8","Type":"ContainerDied","Data":"4babb5839e8b3ac96725738008c894105d2676a536e4897b2e08341aa8ca7cd3"}
Oct 30 19:21:28.581240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.581216    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal" event={"ID":"3412174e80de20d346c1e09c794a54c8","Type":"ContainerStarted","Data":"fe16d935efdb60f47b80f91c0cd9440e9da8c1aa652bdf06a543d6ccec9b3708"}
Oct 30 19:21:28.581435 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.581373    5907 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal"
Oct 30 19:21:28.607069 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.606995    5907 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 19:21:28.607069 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:21:28.607048    5907 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-132-152.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-152.ec2.internal"
Oct 30 19:21:28.613021 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.613001    5907 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 19:21:28.613100 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:21:28.613038    5907 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal"
Oct 30 19:21:28.693404 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.693365    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e58ff4b1131d98fcba47fc4e4f2a3553-config\") pod \"kube-apiserver-proxy-ip-10-0-132-152.ec2.internal\" (UID: \"e58ff4b1131d98fcba47fc4e4f2a3553\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-152.ec2.internal"
Oct 30 19:21:28.693404 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.693406    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3412174e80de20d346c1e09c794a54c8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal\" (UID: \"3412174e80de20d346c1e09c794a54c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal"
Oct 30 19:21:28.693639 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.693437    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3412174e80de20d346c1e09c794a54c8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal\" (UID: \"3412174e80de20d346c1e09c794a54c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal"
Oct 30 19:21:28.794005 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.793960    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e58ff4b1131d98fcba47fc4e4f2a3553-config\") pod \"kube-apiserver-proxy-ip-10-0-132-152.ec2.internal\" (UID: \"e58ff4b1131d98fcba47fc4e4f2a3553\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-152.ec2.internal"
Oct 30 19:21:28.794005 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.794006    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3412174e80de20d346c1e09c794a54c8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal\" (UID: \"3412174e80de20d346c1e09c794a54c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal"
Oct 30 19:21:28.794239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.794025    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3412174e80de20d346c1e09c794a54c8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal\" (UID: \"3412174e80de20d346c1e09c794a54c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal"
Oct 30 19:21:28.794239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.794082    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3412174e80de20d346c1e09c794a54c8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal\" (UID: \"3412174e80de20d346c1e09c794a54c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal"
Oct 30 19:21:28.794239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.794086    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e58ff4b1131d98fcba47fc4e4f2a3553-config\") pod \"kube-apiserver-proxy-ip-10-0-132-152.ec2.internal\" (UID: \"e58ff4b1131d98fcba47fc4e4f2a3553\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-152.ec2.internal"
Oct 30 19:21:28.794239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:28.794082    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3412174e80de20d346c1e09c794a54c8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal\" (UID: \"3412174e80de20d346c1e09c794a54c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal"
Oct 30 19:21:29.225373 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.225333    5907 apiserver.go:52] "Watching apiserver"
Oct 30 19:21:29.283897 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.283847    5907 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 19:21:29.284717 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.284691    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-jqs6k","openshift-multus/network-metrics-daemon-9jk99","openshift-network-operator/iptables-alerter-c9vlh","kube-system/kube-apiserver-proxy-ip-10-0-132-152.ec2.internal","openshift-monitoring/alertmanager-main-0","openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh","openshift-multus/multus-additional-cni-plugins-hcmmq","openshift-monitoring/prometheus-k8s-0","kube-system/konnectivity-agent-nw4jx","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch","openshift-console/console-6bdc9bd9c6-mww4x","openshift-insights/insights-runtime-extractor-p8nq5","openshift-console/console-5d79db5f98-fmng5","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal","openshift-network-diagnostics/network-check-target-5j799","openshift-image-registry/node-ca-tgt5g","openshift-dns/node-resolver-f7tbh","openshift-ingress-canary/ingress-canary-q2cc6","openshift-monitoring/monitoring-plugin-77f4f76594-mffps","openshift-monitoring/openshift-state-metrics-7bd7489957-8k92x","openshift-monitoring/thanos-querier-66d96d76c5-nmsds","openshift-ovn-kubernetes/ovnkube-node-8pv94","openshift-cluster-node-tuning-operator/tuned-dvzwm","openshift-monitoring/metrics-server-6b87cd958c-prt2j","kube-system/global-pull-secret-syncer-z7mhx","openshift-dns/dns-default-vn5fp","openshift-multus/multus-sdlcz"]
Oct 30 19:21:29.301331 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.301297    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 19:21:29.301331 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.301297    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 19:21:29.301566 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.301468    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 19:21:29.301882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.301839    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 19:21:29.301989 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.301973    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 19:21:29.302333 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.302316    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 19:21:29.302534 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.302516    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 19:21:29.302582 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.302558    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 19:21:29.303416 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.303395    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 19:21:29.303520 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.303433    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 19:21:29.303520 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.303472    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 19:21:29.304530 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.304516    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 19:21:29.314796 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.314775    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 19:21:29.315019 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.315003    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 19:21:29.316003 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.315978    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 19:21:29.316003 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.315999    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 19:21:29.316145 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.316043    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 19:21:29.316145 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.316078    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 19:21:29.316145 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.316134    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 19:21:29.316260 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.316140    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 30 19:21:29.316366 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.316353    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 19:21:29.316580 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.316559    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8rhbj\""
Oct 30 19:21:29.316580 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.316560    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-9smtf\""
Oct 30 19:21:29.316722 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.316560    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gx26w\""
Oct 30 19:21:29.316722 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.316604    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gqsl5\""
Oct 30 19:21:29.316722 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.316621    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sl59z\""
Oct 30 19:21:29.316722 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.316632    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-65nsn\""
Oct 30 19:21:29.316722 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.316644    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 19:21:29.316722 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.316636    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bkrzr\""
Oct 30 19:21:29.316722 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.316634    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-m2qt2\""
Oct 30 19:21:29.316722 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.316674    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-w8vq8\""
Oct 30 19:21:29.316722 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.316682    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-b7kzg\""
Oct 30 19:21:29.316996 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.316906    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-gx5x8\""
Oct 30 19:21:29.317426 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.317411    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 19:21:29.318634 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.318620    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 19:21:29.319067 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.319055    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 19:21:29.319159 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.319148    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 19:21:29.319272 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.319259    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 19:21:29.319308 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.319299    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 19:21:29.319408 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.319394    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Oct 30 19:21:29.319622 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.319608    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 19:21:29.319674 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.319625    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 19:21:29.319674 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.319645    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 19:21:29.319674 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.319659    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 19:21:29.319781 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.319698    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 19:21:29.349475 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.349438    5907 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 19:21:29.356451 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.356429    5907 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-152.ec2.internal"
Oct 30 19:21:29.356576 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.356565    5907 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal"
Oct 30 19:21:29.379614 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.379581    5907 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 19:21:29.379833 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.379625    5907 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 19:21:29.379833 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:21:29.379643    5907 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-152.ec2.internal"
Oct 30 19:21:29.379833 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:21:29.379667    5907 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-132-152.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-152.ec2.internal"
Oct 30 19:21:29.396219 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.396192    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2ca12196-1929-4ece-a076-d36527b95ff1-metrics-server-audit-profiles\") pod \"metrics-server-6b87cd958c-prt2j\" (UID: \"2ca12196-1929-4ece-a076-d36527b95ff1\") " pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:29.396278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.396222    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-tmp\") pod \"console-5d79db5f98-fmng5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") " pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:29.396278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.396243    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.396278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.396260    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1fda3255-2b22-40a0-870b-7b940977e197-multus-daemon-config\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.396422 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.396276    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-console-config\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:29.396422 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.396327    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/57769216-dd32-4edc-8e73-b4873caa041d-host-slash\") pod \"iptables-alerter-c9vlh\" (UID: \"57769216-dd32-4edc-8e73-b4873caa041d\") " pod="openshift-network-operator/iptables-alerter-c9vlh"
Oct 30 19:21:29.396422 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.396353    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/725d2cce-2a75-426f-8522-d1ddfbabe1b6-dbus\") pod \"global-pull-secret-syncer-z7mhx\" (UID: \"725d2cce-2a75-426f-8522-d1ddfbabe1b6\") " pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:21:29.396422 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.396367    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-tmp\") pod \"console-5d79db5f98-fmng5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") " pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:29.396586 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.396425    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-slash\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.396586 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.396453    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-log-socket\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.396586 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.396502    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ffad3bc-f14e-4132-b5ee-3ec84d009252-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-mlpqh\" (UID: \"1ffad3bc-f14e-4132-b5ee-3ec84d009252\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh"
Oct 30 19:21:29.396586 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.396517    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1fda3255-2b22-40a0-870b-7b940977e197-multus-daemon-config\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.396586 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.396537    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-sys\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.396586 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.396565    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c59e8b44-50b7-4a50-af6a-1bb9a7547a56-cert\") pod \"ingress-canary-q2cc6\" (UID: \"c59e8b44-50b7-4a50-af6a-1bb9a7547a56\") " pod="openshift-ingress-canary/ingress-canary-q2cc6"
Oct 30 19:21:29.396759 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.396600    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-trusted-ca-bundle\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:29.396759 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.396640    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1ceff922-50ab-4e0f-b113-03719eb40bc9-cnibin\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:21:29.396759 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.396671    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/725d2cce-2a75-426f-8522-d1ddfbabe1b6-kubelet-config\") pod \"global-pull-secret-syncer-z7mhx\" (UID: \"725d2cce-2a75-426f-8522-d1ddfbabe1b6\") " pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:21:29.396759 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.396696    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-systemd-units\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.396759 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.396726    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.397030 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.396788    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-system-cni-dir\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.397030 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.396830    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b70ef66-ed5b-4917-8bca-36757fee37e3-tmp-dir\") pod \"dns-default-vn5fp\" (UID: \"4b70ef66-ed5b-4917-8bca-36757fee37e3\") " pod="openshift-dns/dns-default-vn5fp"
Oct 30 19:21:29.397030 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.396836    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c59e8b44-50b7-4a50-af6a-1bb9a7547a56-cert\") pod \"ingress-canary-q2cc6\" (UID: \"c59e8b44-50b7-4a50-af6a-1bb9a7547a56\") " pod="openshift-ingress-canary/ingress-canary-q2cc6"
Oct 30 19:21:29.397030 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.396903    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-ovn-node-metrics-cert\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.397030 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.396926    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b70ef66-ed5b-4917-8bca-36757fee37e3-tmp-dir\") pod \"dns-default-vn5fp\" (UID: \"4b70ef66-ed5b-4917-8bca-36757fee37e3\") " pod="openshift-dns/dns-default-vn5fp"
Oct 30 19:21:29.397030 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.396932    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-run-netns\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.397030 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.396964    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/40121914-4632-47b7-bc42-8299b3d6d6bd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:29.397030 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.396996    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.397030 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397025    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ffad3bc-f14e-4132-b5ee-3ec84d009252-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-mlpqh\" (UID: \"1ffad3bc-f14e-4132-b5ee-3ec84d009252\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh"
Oct 30 19:21:29.397331 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397056    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-lib-modules\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.397331 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397068    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/40121914-4632-47b7-bc42-8299b3d6d6bd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:29.397331 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397082    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-console-serving-cert\") pod \"console-5d79db5f98-fmng5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") " pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:29.397331 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397107    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-cnibin\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.397331 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397137    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-ovn-node-metrics-cert\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.397331 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397145    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-run-multus-certs\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.397331 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397178    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-etc-kubernetes\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.397331 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397203    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg2rm\" (UniqueName: \"kubernetes.io/projected/41ff5ee2-70a2-4baf-af4e-00e6575c2e99-kube-api-access-rg2rm\") pod \"network-metrics-daemon-9jk99\" (UID: \"41ff5ee2-70a2-4baf-af4e-00e6575c2e99\") " pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:21:29.397331 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397244    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ffad3bc-f14e-4132-b5ee-3ec84d009252-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-mlpqh\" (UID: \"1ffad3bc-f14e-4132-b5ee-3ec84d009252\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh"
Oct 30 19:21:29.397331 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397245    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f1622f2e-319b-4259-b314-a4178b7875f0-node-exporter-textfile\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:29.397331 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397284    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmftk\" (UniqueName: \"kubernetes.io/projected/f1622f2e-319b-4259-b314-a4178b7875f0-kube-api-access-wmftk\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:29.397331 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397322    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f1622f2e-319b-4259-b314-a4178b7875f0-node-exporter-textfile\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:29.397819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397361    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-grpc-tls\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:29.397819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397379    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.397819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397397    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:29.397819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397421    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1ceff922-50ab-4e0f-b113-03719eb40bc9-cni-binary-copy\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:21:29.397819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397456    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9a1d2cec-f539-4106-a185-3134a3df747c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p8nq5\" (UID: \"9a1d2cec-f539-4106-a185-3134a3df747c\") " pod="openshift-insights/insights-runtime-extractor-p8nq5"
Oct 30 19:21:29.397819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397485    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a4ed12fa-0026-4bf8-a99c-83b04caac774-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-8k92x\" (UID: \"a4ed12fa-0026-4bf8-a99c-83b04caac774\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8k92x"
Oct 30 19:21:29.397819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397515    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f1622f2e-319b-4259-b314-a4178b7875f0-root\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:29.397819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397540    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b70ef66-ed5b-4917-8bca-36757fee37e3-metrics-tls\") pod \"dns-default-vn5fp\" (UID: \"4b70ef66-ed5b-4917-8bca-36757fee37e3\") " pod="openshift-dns/dns-default-vn5fp"
Oct 30 19:21:29.397819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397562    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:29.397819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397592    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2c840c6b-6992-4f29-9023-a0ad99fdc543-agent-certs\") pod \"konnectivity-agent-nw4jx\" (UID: \"2c840c6b-6992-4f29-9023-a0ad99fdc543\") " pod="kube-system/konnectivity-agent-nw4jx"
Oct 30 19:21:29.397819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397612    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1622f2e-319b-4259-b314-a4178b7875f0-metrics-client-ca\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:29.397819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397636    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1ffad3bc-f14e-4132-b5ee-3ec84d009252-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-mlpqh\" (UID: \"1ffad3bc-f14e-4132-b5ee-3ec84d009252\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh"
Oct 30 19:21:29.397819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397652    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-console-config\") pod \"console-5d79db5f98-fmng5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") " pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:29.397819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397668    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba4ee546-c9c1-4573-b11f-6ba0621972d1-host\") pod \"node-ca-tgt5g\" (UID: \"ba4ee546-c9c1-4573-b11f-6ba0621972d1\") " pod="openshift-image-registry/node-ca-tgt5g"
Oct 30 19:21:29.397819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397671    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a4ed12fa-0026-4bf8-a99c-83b04caac774-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-8k92x\" (UID: \"a4ed12fa-0026-4bf8-a99c-83b04caac774\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8k92x"
Oct 30 19:21:29.397819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397688    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwf55\" (UniqueName: \"kubernetes.io/projected/ba4ee546-c9c1-4573-b11f-6ba0621972d1-kube-api-access-rwf55\") pod \"node-ca-tgt5g\" (UID: \"ba4ee546-c9c1-4573-b11f-6ba0621972d1\") " pod="openshift-image-registry/node-ca-tgt5g"
Oct 30 19:21:29.397819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397716    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1ffad3bc-f14e-4132-b5ee-3ec84d009252-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-mlpqh\" (UID: \"1ffad3bc-f14e-4132-b5ee-3ec84d009252\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh"
Oct 30 19:21:29.397819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397728    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4b70ef66-ed5b-4917-8bca-36757fee37e3-config-volume\") pod \"dns-default-vn5fp\" (UID: \"4b70ef66-ed5b-4917-8bca-36757fee37e3\") " pod="openshift-dns/dns-default-vn5fp"
Oct 30 19:21:29.397819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397765    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/57769216-dd32-4edc-8e73-b4873caa041d-iptables-alerter-script\") pod \"iptables-alerter-c9vlh\" (UID: \"57769216-dd32-4edc-8e73-b4873caa041d\") " pod="openshift-network-operator/iptables-alerter-c9vlh"
Oct 30 19:21:29.397819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397792    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a4ed12fa-0026-4bf8-a99c-83b04caac774-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-8k92x\" (UID: \"a4ed12fa-0026-4bf8-a99c-83b04caac774\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8k92x"
Oct 30 19:21:29.397819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397796    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2c840c6b-6992-4f29-9023-a0ad99fdc543-agent-certs\") pod \"konnectivity-agent-nw4jx\" (UID: \"2c840c6b-6992-4f29-9023-a0ad99fdc543\") " pod="kube-system/konnectivity-agent-nw4jx"
Oct 30 19:21:29.397819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397813    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1622f2e-319b-4259-b314-a4178b7875f0-metrics-client-ca\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:29.397819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397821    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-etc-openvswitch\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.398803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397850    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-run-ovn-kubernetes\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.398803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397910    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4b70ef66-ed5b-4917-8bca-36757fee37e3-config-volume\") pod \"dns-default-vn5fp\" (UID: \"4b70ef66-ed5b-4917-8bca-36757fee37e3\") " pod="openshift-dns/dns-default-vn5fp"
Oct 30 19:21:29.398803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397923    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b41606bf-e8ca-4fc6-8805-b7461187d65e-tmp-dir\") pod \"node-resolver-f7tbh\" (UID: \"b41606bf-e8ca-4fc6-8805-b7461187d65e\") " pod="openshift-dns/node-resolver-f7tbh"
Oct 30 19:21:29.398803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397957    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-run\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.398803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397977    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/57769216-dd32-4edc-8e73-b4873caa041d-iptables-alerter-script\") pod \"iptables-alerter-c9vlh\" (UID: \"57769216-dd32-4edc-8e73-b4873caa041d\") " pod="openshift-network-operator/iptables-alerter-c9vlh"
Oct 30 19:21:29.398803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.397997    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b41606bf-e8ca-4fc6-8805-b7461187d65e-tmp-dir\") pod \"node-resolver-f7tbh\" (UID: \"b41606bf-e8ca-4fc6-8805-b7461187d65e\") " pod="openshift-dns/node-resolver-f7tbh"
Oct 30 19:21:29.398803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.398023    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.398803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.398057    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-web-config\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.398803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.398083    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2ca12196-1929-4ece-a076-d36527b95ff1-audit-log\") pod \"metrics-server-6b87cd958c-prt2j\" (UID: \"2ca12196-1929-4ece-a076-d36527b95ff1\") " pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:29.398803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.398112    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-modprobe-d\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.398803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.398145    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40121914-4632-47b7-bc42-8299b3d6d6bd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:29.398803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.398171    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2ca12196-1929-4ece-a076-d36527b95ff1-audit-log\") pod \"metrics-server-6b87cd958c-prt2j\" (UID: \"2ca12196-1929-4ece-a076-d36527b95ff1\") " pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:29.398803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.398171    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-run-ovn\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.398803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.398227    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-multus-conf-dir\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.398803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.398246    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40121914-4632-47b7-bc42-8299b3d6d6bd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:29.398803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.398262    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-console-serving-cert\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:29.398803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.398313    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-tmp\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:29.398803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.398414    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40121914-4632-47b7-bc42-8299b3d6d6bd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:29.398803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.398415    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1ceff922-50ab-4e0f-b113-03719eb40bc9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:21:29.398803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.398437    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skgdt\" (UniqueName: \"kubernetes.io/projected/1ceff922-50ab-4e0f-b113-03719eb40bc9-kube-api-access-skgdt\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:21:29.398803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.398459    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-tmp\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:29.398803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.398499    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2c840c6b-6992-4f29-9023-a0ad99fdc543-konnectivity-ca\") pod \"konnectivity-agent-nw4jx\" (UID: \"2c840c6b-6992-4f29-9023-a0ad99fdc543\") " pod="kube-system/konnectivity-agent-nw4jx"
Oct 30 19:21:29.398803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.398526    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-node-log\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.398803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.398554    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.398803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.398597    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.398803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.398621    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rvqr\" (UniqueName: \"kubernetes.io/projected/57769216-dd32-4edc-8e73-b4873caa041d-kube-api-access-4rvqr\") pod \"iptables-alerter-c9vlh\" (UID: \"57769216-dd32-4edc-8e73-b4873caa041d\") " pod="openshift-network-operator/iptables-alerter-c9vlh"
Oct 30 19:21:29.398803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.398639    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2xwk\" (UniqueName: \"kubernetes.io/projected/cd444ba1-edf1-49aa-a0cf-b863c93e60be-kube-api-access-c2xwk\") pod \"network-check-target-5j799\" (UID: \"cd444ba1-edf1-49aa-a0cf-b863c93e60be\") " pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:21:29.398803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.398646    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1ceff922-50ab-4e0f-b113-03719eb40bc9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:21:29.398803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.398659    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-run-netns\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.398803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.398686    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2c840c6b-6992-4f29-9023-a0ad99fdc543-konnectivity-ca\") pod \"konnectivity-agent-nw4jx\" (UID: \"2c840c6b-6992-4f29-9023-a0ad99fdc543\") " pod="kube-system/konnectivity-agent-nw4jx"
Oct 30 19:21:29.398803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.398777    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bln4w\" (UniqueName: \"kubernetes.io/projected/b41606bf-e8ca-4fc6-8805-b7461187d65e-kube-api-access-bln4w\") pod \"node-resolver-f7tbh\" (UID: \"b41606bf-e8ca-4fc6-8805-b7461187d65e\") " pod="openshift-dns/node-resolver-f7tbh"
Oct 30 19:21:29.398803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.398795    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b05813e-ce3d-4e71-a64e-7417997c2988-tmp\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:29.398803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.398811    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-kubernetes\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.398803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.398828    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8635205-1f61-4a17-bc1f-7876c5536f9c-tmp\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.398842    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-multus-cni-dir\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.398898    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k9b2\" (UniqueName: \"kubernetes.io/projected/9a1d2cec-f539-4106-a185-3134a3df747c-kube-api-access-8k9b2\") pod \"insights-runtime-extractor-p8nq5\" (UID: \"9a1d2cec-f539-4106-a185-3134a3df747c\") " pod="openshift-insights/insights-runtime-extractor-p8nq5"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.398924    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b05813e-ce3d-4e71-a64e-7417997c2988-tmp\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.398927    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8635205-1f61-4a17-bc1f-7876c5536f9c-tmp\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.398990    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3b05813e-ce3d-4e71-a64e-7417997c2988-metrics-client-ca\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.399079    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9wwz\" (UniqueName: \"kubernetes.io/projected/aa367dd7-da29-424e-9acc-7706f4377494-kube-api-access-s9wwz\") pod \"monitoring-plugin-77f4f76594-mffps\" (UID: \"aa367dd7-da29-424e-9acc-7706f4377494\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-mffps"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.399133    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2ca12196-1929-4ece-a076-d36527b95ff1-secret-metrics-server-tls\") pod \"metrics-server-6b87cd958c-prt2j\" (UID: \"2ca12196-1929-4ece-a076-d36527b95ff1\") " pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.399198    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.399207    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3b05813e-ce3d-4e71-a64e-7417997c2988-metrics-client-ca\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.399258    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqn8c\" (UniqueName: \"kubernetes.io/projected/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-kube-api-access-pqn8c\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.399320    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1fda3255-2b22-40a0-870b-7b940977e197-cni-binary-copy\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.399345    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-device-dir\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.399368    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-cni-bin\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.399388    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqrkv\" (UniqueName: \"kubernetes.io/projected/3b05813e-ce3d-4e71-a64e-7417997c2988-kube-api-access-jqrkv\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.399409    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-var-lib-kubelet\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.399438    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.399577    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.399579    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-var-lib-cni-multus\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.399633    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-var-lib-kubelet\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.399701    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-service-ca\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.399744    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-oauth-serving-cert\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.399780    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9a1d2cec-f539-4106-a185-3134a3df747c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p8nq5\" (UID: \"9a1d2cec-f539-4106-a185-3134a3df747c\") " pod="openshift-insights/insights-runtime-extractor-p8nq5"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.399811    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dls7\" (UniqueName: \"kubernetes.io/projected/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-kube-api-access-5dls7\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.399839    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f1622f2e-319b-4259-b314-a4178b7875f0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.399905    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f1622f2e-319b-4259-b314-a4178b7875f0-node-exporter-accelerators-collector-config\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.399934    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1ffad3bc-f14e-4132-b5ee-3ec84d009252-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-mlpqh\" (UID: \"1ffad3bc-f14e-4132-b5ee-3ec84d009252\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.399983    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2ca12196-1929-4ece-a076-d36527b95ff1-secret-metrics-server-client-certs\") pod \"metrics-server-6b87cd958c-prt2j\" (UID: \"2ca12196-1929-4ece-a076-d36527b95ff1\") " pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400010    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6klk6\" (UniqueName: \"kubernetes.io/projected/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-kube-api-access-6klk6\") pod \"console-5d79db5f98-fmng5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") " pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400036    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-env-overrides\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400066    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400099    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jdxg\" (UniqueName: \"kubernetes.io/projected/1fda3255-2b22-40a0-870b-7b940977e197-kube-api-access-5jdxg\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400116    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f1622f2e-319b-4259-b314-a4178b7875f0-node-exporter-accelerators-collector-config\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400125    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-console-oauth-config\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400152    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-ovnkube-script-lib\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400197    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-web-config\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400243    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-ovnkube-config\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400243    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1ffad3bc-f14e-4132-b5ee-3ec84d009252-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-mlpqh\" (UID: \"1ffad3bc-f14e-4132-b5ee-3ec84d009252\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh"
Oct 30 19:21:29.400278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400262    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-env-overrides\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400369    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f1622f2e-319b-4259-b314-a4178b7875f0-node-exporter-wtmp\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400402    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-sysctl-d\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400429    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-var-lib-cni-bin\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400476    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-ovnkube-script-lib\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400499    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ba4ee546-c9c1-4573-b11f-6ba0621972d1-serviceca\") pod \"node-ca-tgt5g\" (UID: \"ba4ee546-c9c1-4573-b11f-6ba0621972d1\") " pod="openshift-image-registry/node-ca-tgt5g"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400502    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-ovnkube-config\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400529    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400547    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ceff922-50ab-4e0f-b113-03719eb40bc9-system-cni-dir\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400564    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9a1d2cec-f539-4106-a185-3134a3df747c-crio-socket\") pod \"insights-runtime-extractor-p8nq5\" (UID: \"9a1d2cec-f539-4106-a185-3134a3df747c\") " pod="openshift-insights/insights-runtime-extractor-p8nq5"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400592    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400617    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-console-oauth-config\") pod \"console-5d79db5f98-fmng5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") " pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400635    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400653    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-sys-fs\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400670    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv49j\" (UniqueName: \"kubernetes.io/projected/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-kube-api-access-vv49j\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400686    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a1d2cec-f539-4106-a185-3134a3df747c-data-volume\") pod \"insights-runtime-extractor-p8nq5\" (UID: \"9a1d2cec-f539-4106-a185-3134a3df747c\") " pod="openshift-insights/insights-runtime-extractor-p8nq5"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400704    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4ed12fa-0026-4bf8-a99c-83b04caac774-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-8k92x\" (UID: \"a4ed12fa-0026-4bf8-a99c-83b04caac774\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8k92x"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400741    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd95d\" (UniqueName: \"kubernetes.io/projected/a4ed12fa-0026-4bf8-a99c-83b04caac774-kube-api-access-rd95d\") pod \"openshift-state-metrics-7bd7489957-8k92x\" (UID: \"a4ed12fa-0026-4bf8-a99c-83b04caac774\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8k92x"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400759    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ca12196-1929-4ece-a076-d36527b95ff1-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6b87cd958c-prt2j\" (UID: \"2ca12196-1929-4ece-a076-d36527b95ff1\") " pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400775    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-service-ca\") pod \"console-5d79db5f98-fmng5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") " pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400793    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1ceff922-50ab-4e0f-b113-03719eb40bc9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400810    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1ceff922-50ab-4e0f-b113-03719eb40bc9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400826    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-kubelet\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400841    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-config-out\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400883    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400910    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.400996    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hcrs\" (UniqueName: \"kubernetes.io/projected/8161e358-9105-4d99-88c1-98c17db09674-kube-api-access-4hcrs\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.401137    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1ceff922-50ab-4e0f-b113-03719eb40bc9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.401139    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vlf5\" (UniqueName: \"kubernetes.io/projected/40121914-4632-47b7-bc42-8299b3d6d6bd-kube-api-access-6vlf5\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.401184    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-run-openvswitch\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.401218    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ffad3bc-f14e-4132-b5ee-3ec84d009252-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-mlpqh\" (UID: \"1ffad3bc-f14e-4132-b5ee-3ec84d009252\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.401258    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.401288    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ba4ee546-c9c1-4573-b11f-6ba0621972d1-serviceca\") pod \"node-ca-tgt5g\" (UID: \"ba4ee546-c9c1-4573-b11f-6ba0621972d1\") " pod="openshift-image-registry/node-ca-tgt5g"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.401295    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a1d2cec-f539-4106-a185-3134a3df747c-data-volume\") pod \"insights-runtime-extractor-p8nq5\" (UID: \"9a1d2cec-f539-4106-a185-3134a3df747c\") " pod="openshift-insights/insights-runtime-extractor-p8nq5"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.401413    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdsrm\" (UniqueName: \"kubernetes.io/projected/1ffad3bc-f14e-4132-b5ee-3ec84d009252-kube-api-access-mdsrm\") pod \"kube-state-metrics-5dc8d7d89b-mlpqh\" (UID: \"1ffad3bc-f14e-4132-b5ee-3ec84d009252\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.401466    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-sysconfig\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.401504    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4ed12fa-0026-4bf8-a99c-83b04caac774-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-8k92x\" (UID: \"a4ed12fa-0026-4bf8-a99c-83b04caac774\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8k92x"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.401552    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5c8j\" (UniqueName: \"kubernetes.io/projected/c59e8b44-50b7-4a50-af6a-1bb9a7547a56-kube-api-access-t5c8j\") pod \"ingress-canary-q2cc6\" (UID: \"c59e8b44-50b7-4a50-af6a-1bb9a7547a56\") " pod="openshift-ingress-canary/ingress-canary-q2cc6"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.401595    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/725d2cce-2a75-426f-8522-d1ddfbabe1b6-original-pull-secret\") pod \"global-pull-secret-syncer-z7mhx\" (UID: \"725d2cce-2a75-426f-8522-d1ddfbabe1b6\") " pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.401635    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-sysctl-conf\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.401666    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-hostroot\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.401690    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-config-out\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.401697    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/40121914-4632-47b7-bc42-8299b3d6d6bd-config-out\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.401758    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/40121914-4632-47b7-bc42-8299b3d6d6bd-config-out\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.401786    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41ff5ee2-70a2-4baf-af4e-00e6575c2e99-metrics-certs\") pod \"network-metrics-daemon-9jk99\" (UID: \"41ff5ee2-70a2-4baf-af4e-00e6575c2e99\") " pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.401815    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1622f2e-319b-4259-b314-a4178b7875f0-sys\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.401843    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txjsq\" (UniqueName: \"kubernetes.io/projected/a8635205-1f61-4a17-bc1f-7876c5536f9c-kube-api-access-txjsq\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.401881    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.401884    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.401911    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.401959    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-run-k8s-cni-cncf-io\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.401937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.401974    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.401984    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.401987    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/40121914-4632-47b7-bc42-8299b3d6d6bd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.402040    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.402107    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.402040    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2ca12196-1929-4ece-a076-d36527b95ff1-client-ca-bundle\") pod \"metrics-server-6b87cd958c-prt2j\" (UID: \"2ca12196-1929-4ece-a076-d36527b95ff1\") " pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.402147    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.402171    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.402174    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.402201    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-tuned\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.402227    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-config\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.402243    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.402253    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-cni-netd\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.402298    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-thanos-querier-tls\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.402317    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-tuned\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.402349    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.402372    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/aa367dd7-da29-424e-9acc-7706f4377494-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-mffps\" (UID: \"aa367dd7-da29-424e-9acc-7706f4377494\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-mffps"
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.402397    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.402402    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-systemd\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.402474    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.402509    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.402512    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-socket-dir\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.402538    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.402541    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-run-systemd\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.402572    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f1622f2e-319b-4259-b314-a4178b7875f0-node-exporter-tls\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.402637    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slnw6\" (UniqueName: \"kubernetes.io/projected/2ca12196-1929-4ece-a076-d36527b95ff1-kube-api-access-slnw6\") pod \"metrics-server-6b87cd958c-prt2j\" (UID: \"2ca12196-1929-4ece-a076-d36527b95ff1\") " pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.402674    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.402705    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-etc-selinux\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.402739    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvcz2\" (UniqueName: \"kubernetes.io/projected/4b70ef66-ed5b-4917-8bca-36757fee37e3-kube-api-access-mvcz2\") pod \"dns-default-vn5fp\" (UID: \"4b70ef66-ed5b-4917-8bca-36757fee37e3\") " pod="openshift-dns/dns-default-vn5fp"
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.402784    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.402786    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-config-volume\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.402832    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.402892    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.402927    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.402935    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.402925    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-multus-socket-dir-parent\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.402978    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.403005    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.403059    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-oauth-serving-cert\") pod \"console-5d79db5f98-fmng5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") " pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.403093    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b41606bf-e8ca-4fc6-8805-b7461187d65e-hosts-file\") pod \"node-resolver-f7tbh\" (UID: \"b41606bf-e8ca-4fc6-8805-b7461187d65e\") " pod="openshift-dns/node-resolver-f7tbh"
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.403107    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.403133    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.403143    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.403166    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-host\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.403194    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-registration-dir\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.403222    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.403247    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.403264    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-var-lib-openvswitch\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.404094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.403428    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 30 19:21:29.407526 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.407506    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-console-config\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:29.407607 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.407525    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.407607 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.407595    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2ca12196-1929-4ece-a076-d36527b95ff1-metrics-server-audit-profiles\") pod \"metrics-server-6b87cd958c-prt2j\" (UID: \"2ca12196-1929-4ece-a076-d36527b95ff1\") " pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:29.407716 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.407622    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1ceff922-50ab-4e0f-b113-03719eb40bc9-cni-binary-copy\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:21:29.407716 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.407676    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9a1d2cec-f539-4106-a185-3134a3df747c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p8nq5\" (UID: \"9a1d2cec-f539-4106-a185-3134a3df747c\") " pod="openshift-insights/insights-runtime-extractor-p8nq5"
Oct 30 19:21:29.407944 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.407925    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-console-config\") pod \"console-5d79db5f98-fmng5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") " pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:29.409091 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.409077    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.409937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.409918    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1fda3255-2b22-40a0-870b-7b940977e197-cni-binary-copy\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.410005 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.409965    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-service-ca\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:29.410005 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.409965    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-oauth-serving-cert\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:29.411164 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.411147    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ca12196-1929-4ece-a076-d36527b95ff1-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6b87cd958c-prt2j\" (UID: \"2ca12196-1929-4ece-a076-d36527b95ff1\") " pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:29.411336 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.411313    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-service-ca\") pod \"console-5d79db5f98-fmng5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") " pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:29.413329 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.413314    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-oauth-serving-cert\") pod \"console-5d79db5f98-fmng5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") " pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:29.413437 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.413346    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.417265 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.417238    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 19:21:29.417657 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.417634    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 19:21:29.417757 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.417735    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Oct 30 19:21:29.417810 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.417771    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 19:21:29.417883 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.417834    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 19:21:29.417947 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.417836    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 30 19:21:29.417947 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.417910    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 19:21:29.417947 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.417910    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 19:21:29.418074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.417948    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 19:21:29.418074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.417965    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Oct 30 19:21:29.418074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.417998    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 19:21:29.418074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.418006    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Oct 30 19:21:29.418074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.418027    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 19:21:29.418074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.418043    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 19:21:29.418074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.418045    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 30 19:21:29.418376 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.418099    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 19:21:29.418376 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.418111    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 19:21:29.418376 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.418125    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 19:21:29.418376 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.418116    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 19:21:29.418376 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.418150    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Oct 30 19:21:29.418376 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.418152    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 19:21:29.418376 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.418113    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 19:21:29.418376 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.418178    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 19:21:29.418376 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.418218    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Oct 30 19:21:29.418376 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.418173    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Oct 30 19:21:29.418376 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.418377    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7380srhp40v4j\""
Oct 30 19:21:29.418803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.418375    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-web-config\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.418803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.418491    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 19:21:29.418803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.418541    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40121914-4632-47b7-bc42-8299b3d6d6bd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:29.418803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.418544    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Oct 30 19:21:29.421924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.419182    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg2rm\" (UniqueName: \"kubernetes.io/projected/41ff5ee2-70a2-4baf-af4e-00e6575c2e99-kube-api-access-rg2rm\") pod \"network-metrics-daemon-9jk99\" (UID: \"41ff5ee2-70a2-4baf-af4e-00e6575c2e99\") " pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:21:29.421924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.419184    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.421924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.419316    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Oct 30 19:21:29.421924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.419501    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Oct 30 19:21:29.421924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.419520    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.421924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.419557    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-qh4dk\""
Oct 30 19:21:29.421924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.419667    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-h5vgg\""
Oct 30 19:21:29.421924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.419753    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmftk\" (UniqueName: \"kubernetes.io/projected/f1622f2e-319b-4259-b314-a4178b7875f0-kube-api-access-wmftk\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:29.421924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.420011    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-2nv6s\""
Oct 30 19:21:29.421924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.420019    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 19:21:29.421924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.419318    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-xvh7n\""
Oct 30 19:21:29.421924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.420360    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f1622f2e-319b-4259-b314-a4178b7875f0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:29.421924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.420496    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 19:21:29.421924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.420580    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hxbtq\""
Oct 30 19:21:29.421924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.420679    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-console-oauth-config\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:29.421924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.420888    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-4roo0a8peihja\""
Oct 30 19:21:29.421924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.421141    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 19:21:29.421924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.421160    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:29.421924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.421442    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 30 19:21:29.421924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.421475    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-console-oauth-config\") pod \"console-5d79db5f98-fmng5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") " pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:29.421924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.421552    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-77hs9\""
Oct 30 19:21:29.421924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.421717    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 19:21:29.421924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.421746    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ffad3bc-f14e-4132-b5ee-3ec84d009252-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-mlpqh\" (UID: \"1ffad3bc-f14e-4132-b5ee-3ec84d009252\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh"
Oct 30 19:21:29.423080 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.422052    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 30 19:21:29.423080 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.422169    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-cqcj8\""
Oct 30 19:21:29.423080 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.422518    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 19:21:29.423080 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.422741    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Oct 30 19:21:29.423269 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.423165    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.423269 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.423238    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-config\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.423372 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.423312    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.423372 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.423367    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 19:21:29.425008 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.423543    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/aa367dd7-da29-424e-9acc-7706f4377494-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-mffps\" (UID: \"aa367dd7-da29-424e-9acc-7706f4377494\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-mffps"
Oct 30 19:21:29.425008 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.423563    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 19:21:29.425008 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.423676    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 19:21:29.425008 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.423801    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f1622f2e-319b-4259-b314-a4178b7875f0-node-exporter-tls\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:29.425008 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.423822    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:29.425008 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.423875    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-3hcakgtj1qoqn\""
Oct 30 19:21:29.425008 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.423890    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bcv2g\""
Oct 30 19:21:29.425008 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.424130    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-xfcg7\""
Oct 30 19:21:29.425008 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.424363    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-mbz98\""
Oct 30 19:21:29.425008 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.424435    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.425008 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.424440    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41ff5ee2-70a2-4baf-af4e-00e6575c2e99-metrics-certs\") pod \"network-metrics-daemon-9jk99\" (UID: \"41ff5ee2-70a2-4baf-af4e-00e6575c2e99\") " pod="openshift-multus/network-metrics-daemon-9jk99"
Oct 30 19:21:29.425008 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.424636    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-rv27s\""
Oct 30 19:21:29.425008 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.424643    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2ca12196-1929-4ece-a076-d36527b95ff1-secret-metrics-server-tls\") pod \"metrics-server-6b87cd958c-prt2j\" (UID: \"2ca12196-1929-4ece-a076-d36527b95ff1\") " pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:29.425008 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.424932    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:29.425716 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.425353    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hcrs\" (UniqueName: \"kubernetes.io/projected/8161e358-9105-4d99-88c1-98c17db09674-kube-api-access-4hcrs\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:21:29.425716 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.425500    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqrkv\" (UniqueName: \"kubernetes.io/projected/3b05813e-ce3d-4e71-a64e-7417997c2988-kube-api-access-jqrkv\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:29.425716 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.425619    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qmnn8\""
Oct 30 19:21:29.425716 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.425636    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:29.425942 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.425716    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k9b2\" (UniqueName: \"kubernetes.io/projected/9a1d2cec-f539-4106-a185-3134a3df747c-kube-api-access-8k9b2\") pod \"insights-runtime-extractor-p8nq5\" (UID: \"9a1d2cec-f539-4106-a185-3134a3df747c\") " pod="openshift-insights/insights-runtime-extractor-p8nq5"
Oct 30 19:21:29.426066 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.426036    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jdxg\" (UniqueName: \"kubernetes.io/projected/1fda3255-2b22-40a0-870b-7b940977e197-kube-api-access-5jdxg\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.426173 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.426151    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-lkdbb\""
Oct 30 19:21:29.426345 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.426323    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9wwz\" (UniqueName: \"kubernetes.io/projected/aa367dd7-da29-424e-9acc-7706f4377494-kube-api-access-s9wwz\") pod \"monitoring-plugin-77f4f76594-mffps\" (UID: \"aa367dd7-da29-424e-9acc-7706f4377494\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-mffps"
Oct 30 19:21:29.426449 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.426389    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-web-config\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:29.426555 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.426529    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rvqr\" (UniqueName: \"kubernetes.io/projected/57769216-dd32-4edc-8e73-b4873caa041d-kube-api-access-4rvqr\") pod \"iptables-alerter-c9vlh\" (UID: \"57769216-dd32-4edc-8e73-b4873caa041d\") " pod="openshift-network-operator/iptables-alerter-c9vlh"
Oct 30 19:21:29.426637 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.426536    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/40121914-4632-47b7-bc42-8299b3d6d6bd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:29.426637 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.426593    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2ca12196-1929-4ece-a076-d36527b95ff1-secret-metrics-server-client-certs\") pod \"metrics-server-6b87cd958c-prt2j\" (UID: \"2ca12196-1929-4ece-a076-d36527b95ff1\") " pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:29.426637 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.426620    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bln4w\" (UniqueName: \"kubernetes.io/projected/b41606bf-e8ca-4fc6-8805-b7461187d65e-kube-api-access-bln4w\") pod \"node-resolver-f7tbh\" (UID: \"b41606bf-e8ca-4fc6-8805-b7461187d65e\") " pod="openshift-dns/node-resolver-f7tbh"
Oct 30 19:21:29.426818 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.426799    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.426902 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.426829    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:29.426902 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.426831    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/725d2cce-2a75-426f-8522-d1ddfbabe1b6-original-pull-secret\") pod \"global-pull-secret-syncer-z7mhx\" (UID: \"725d2cce-2a75-426f-8522-d1ddfbabe1b6\") " pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:21:29.427002 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.426899    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.427002 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.426964    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5c8j\" (UniqueName: \"kubernetes.io/projected/c59e8b44-50b7-4a50-af6a-1bb9a7547a56-kube-api-access-t5c8j\") pod \"ingress-canary-q2cc6\" (UID: \"c59e8b44-50b7-4a50-af6a-1bb9a7547a56\") " pod="openshift-ingress-canary/ingress-canary-q2cc6"
Oct 30 19:21:29.427160 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.427143    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1ffad3bc-f14e-4132-b5ee-3ec84d009252-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-mlpqh\" (UID: \"1ffad3bc-f14e-4132-b5ee-3ec84d009252\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh"
Oct 30 19:21:29.427420 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.427395    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.427731 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.427691    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-config-volume\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:29.427819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.427764    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-trusted-ca-bundle\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:29.427908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.427833    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b70ef66-ed5b-4917-8bca-36757fee37e3-metrics-tls\") pod \"dns-default-vn5fp\" (UID: \"4b70ef66-ed5b-4917-8bca-36757fee37e3\") " pod="openshift-dns/dns-default-vn5fp"
Oct 30 19:21:29.427908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.427880    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:29.427908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.427901    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:29.428063 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.427923    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdsrm\" (UniqueName: \"kubernetes.io/projected/1ffad3bc-f14e-4132-b5ee-3ec84d009252-kube-api-access-mdsrm\") pod \"kube-state-metrics-5dc8d7d89b-mlpqh\" (UID: \"1ffad3bc-f14e-4132-b5ee-3ec84d009252\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-mlpqh"
Oct 30 19:21:29.428063 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.427935    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.428063 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.427953    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwf55\" (UniqueName: \"kubernetes.io/projected/ba4ee546-c9c1-4573-b11f-6ba0621972d1-kube-api-access-rwf55\") pod \"node-ca-tgt5g\" (UID: \"ba4ee546-c9c1-4573-b11f-6ba0621972d1\") " pod="openshift-image-registry/node-ca-tgt5g"
Oct 30 19:21:29.428063 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.428007    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6klk6\" (UniqueName: \"kubernetes.io/projected/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-kube-api-access-6klk6\") pod \"console-5d79db5f98-fmng5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") " pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:29.428268 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.428072    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.428268 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.428078    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a4ed12fa-0026-4bf8-a99c-83b04caac774-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-8k92x\" (UID: \"a4ed12fa-0026-4bf8-a99c-83b04caac774\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8k92x"
Oct 30 19:21:29.428268 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.427373    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:29.428268 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.428161    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2ca12196-1929-4ece-a076-d36527b95ff1-client-ca-bundle\") pod \"metrics-server-6b87cd958c-prt2j\" (UID: \"2ca12196-1929-4ece-a076-d36527b95ff1\") " pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:29.428268 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.428232    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-console-serving-cert\") pod \"console-5d79db5f98-fmng5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") " pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:29.428618 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.428594    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-skgdt\" (UniqueName: \"kubernetes.io/projected/1ceff922-50ab-4e0f-b113-03719eb40bc9-kube-api-access-skgdt\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:21:29.428676 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.428618    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqn8c\" (UniqueName: \"kubernetes.io/projected/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-kube-api-access-pqn8c\") pod \"prometheus-k8s-0\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:29.428676 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.428632    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dls7\" (UniqueName: \"kubernetes.io/projected/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-kube-api-access-5dls7\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.428760 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.428689    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-slnw6\" (UniqueName: \"kubernetes.io/projected/2ca12196-1929-4ece-a076-d36527b95ff1-kube-api-access-slnw6\") pod \"metrics-server-6b87cd958c-prt2j\" (UID: \"2ca12196-1929-4ece-a076-d36527b95ff1\") " pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:29.428818 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.428805    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-console-serving-cert\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:29.428902 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.428855    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2xwk\" (UniqueName: \"kubernetes.io/projected/cd444ba1-edf1-49aa-a0cf-b863c93e60be-kube-api-access-c2xwk\") pod \"network-check-target-5j799\" (UID: \"cd444ba1-edf1-49aa-a0cf-b863c93e60be\") " pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:21:29.429053 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.429037    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-grpc-tls\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:29.429109 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.429047    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vv49j\" (UniqueName: \"kubernetes.io/projected/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-kube-api-access-vv49j\") pod \"console-6bdc9bd9c6-mww4x\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") " pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:29.429250 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.429233    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txjsq\" (UniqueName: \"kubernetes.io/projected/a8635205-1f61-4a17-bc1f-7876c5536f9c-kube-api-access-txjsq\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.429349 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.429328    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvcz2\" (UniqueName: \"kubernetes.io/projected/4b70ef66-ed5b-4917-8bca-36757fee37e3-kube-api-access-mvcz2\") pod \"dns-default-vn5fp\" (UID: \"4b70ef66-ed5b-4917-8bca-36757fee37e3\") " pod="openshift-dns/dns-default-vn5fp"
Oct 30 19:21:29.429456 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.429431    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rd95d\" (UniqueName: \"kubernetes.io/projected/a4ed12fa-0026-4bf8-a99c-83b04caac774-kube-api-access-rd95d\") pod \"openshift-state-metrics-7bd7489957-8k92x\" (UID: \"a4ed12fa-0026-4bf8-a99c-83b04caac774\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-8k92x"
Oct 30 19:21:29.429540 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.429525    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vlf5\" (UniqueName: \"kubernetes.io/projected/40121914-4632-47b7-bc42-8299b3d6d6bd-kube-api-access-6vlf5\") pod \"alertmanager-main-0\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:29.430138 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.430123    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9a1d2cec-f539-4106-a185-3134a3df747c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p8nq5\" (UID: \"9a1d2cec-f539-4106-a185-3134a3df747c\") " pod="openshift-insights/insights-runtime-extractor-p8nq5"
Oct 30 19:21:29.432681 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.432666    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3b05813e-ce3d-4e71-a64e-7417997c2988-secret-thanos-querier-tls\") pod \"thanos-querier-66d96d76c5-nmsds\" (UID: \"3b05813e-ce3d-4e71-a64e-7417997c2988\") " pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:29.504554 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504463    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-modprobe-d\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.504554 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504504    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-run-ovn\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.504554 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504526    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-multus-conf-dir\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.504554 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504551    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-node-log\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.504851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504575    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-run-netns\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.504851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504595    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-kubernetes\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.504851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504600    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-run-ovn\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.504851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504615    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-multus-cni-dir\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.504851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504600    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-multus-conf-dir\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.504851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504635    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-run-netns\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.504851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504646    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-modprobe-d\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.504851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504646    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-node-log\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.504851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504681    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-multus-cni-dir\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.504851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504680    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-kubernetes\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.504851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504672    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-device-dir\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:21:29.504851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504713    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-device-dir\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:21:29.504851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504734    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-cni-bin\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.504851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504753    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-var-lib-kubelet\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.504851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504774    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-cni-bin\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.504851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504778    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-var-lib-cni-multus\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.504851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504796    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-var-lib-kubelet\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.504851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504809    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-var-lib-kubelet\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.504851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504821    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f1622f2e-319b-4259-b314-a4178b7875f0-node-exporter-wtmp\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:29.504851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504836    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-var-lib-kubelet\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.504851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504835    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-sysctl-d\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.504851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504881    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-var-lib-cni-bin\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504906    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ceff922-50ab-4e0f-b113-03719eb40bc9-system-cni-dir\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504906    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f1622f2e-319b-4259-b314-a4178b7875f0-node-exporter-wtmp\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504922    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9a1d2cec-f539-4106-a185-3134a3df747c-crio-socket\") pod \"insights-runtime-extractor-p8nq5\" (UID: \"9a1d2cec-f539-4106-a185-3134a3df747c\") " pod="openshift-insights/insights-runtime-extractor-p8nq5"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504941    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-sys-fs\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504836    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-var-lib-cni-multus\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504952    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-sysctl-d\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504909    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-var-lib-cni-bin\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504969    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1ceff922-50ab-4e0f-b113-03719eb40bc9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504952    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ceff922-50ab-4e0f-b113-03719eb40bc9-system-cni-dir\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504989    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-sys-fs\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.504995    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-kubelet\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505007    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1ceff922-50ab-4e0f-b113-03719eb40bc9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505022    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505029    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9a1d2cec-f539-4106-a185-3134a3df747c-crio-socket\") pod \"insights-runtime-extractor-p8nq5\" (UID: \"9a1d2cec-f539-4106-a185-3134a3df747c\") " pod="openshift-insights/insights-runtime-extractor-p8nq5"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505044    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-kubelet\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505047    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-run-openvswitch\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505084    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-run-openvswitch\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505100    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505131    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-sysconfig\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505159    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-sysctl-conf\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505180    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-sysconfig\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505182    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-hostroot\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505211    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-hostroot\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505216    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1622f2e-319b-4259-b314-a4178b7875f0-sys\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505234    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1622f2e-319b-4259-b314-a4178b7875f0-sys\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505247    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-run-k8s-cni-cncf-io\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505281    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-sysctl-conf\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505288    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-run-k8s-cni-cncf-io\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505277    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-cni-netd\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505317    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-cni-netd\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505322    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-systemd\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505343    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-socket-dir\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505358    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-run-systemd\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505376    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-etc-selinux\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505402    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-multus-socket-dir-parent\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505407    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-socket-dir\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505424    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b41606bf-e8ca-4fc6-8805-b7461187d65e-hosts-file\") pod \"node-resolver-f7tbh\" (UID: \"b41606bf-e8ca-4fc6-8805-b7461187d65e\") " pod="openshift-dns/node-resolver-f7tbh"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505430    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-run-systemd\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505441    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-etc-selinux\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505456    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-etc-systemd\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505467    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b41606bf-e8ca-4fc6-8805-b7461187d65e-hosts-file\") pod \"node-resolver-f7tbh\" (UID: \"b41606bf-e8ca-4fc6-8805-b7461187d65e\") " pod="openshift-dns/node-resolver-f7tbh"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505474    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-multus-socket-dir-parent\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505497    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-host\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505513    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-registration-dir\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505528    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-var-lib-openvswitch\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505546    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/57769216-dd32-4edc-8e73-b4873caa041d-host-slash\") pod \"iptables-alerter-c9vlh\" (UID: \"57769216-dd32-4edc-8e73-b4873caa041d\") " pod="openshift-network-operator/iptables-alerter-c9vlh"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505560    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/725d2cce-2a75-426f-8522-d1ddfbabe1b6-dbus\") pod \"global-pull-secret-syncer-z7mhx\" (UID: \"725d2cce-2a75-426f-8522-d1ddfbabe1b6\") " pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505576    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-slash\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505579    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-var-lib-openvswitch\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505545    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-host\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505592    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-log-socket\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505621    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-sys\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505624    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-log-socket\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505630    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/725d2cce-2a75-426f-8522-d1ddfbabe1b6-dbus\") pod \"global-pull-secret-syncer-z7mhx\" (UID: \"725d2cce-2a75-426f-8522-d1ddfbabe1b6\") " pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505628    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-slash\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505627    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/57769216-dd32-4edc-8e73-b4873caa041d-host-slash\") pod \"iptables-alerter-c9vlh\" (UID: \"57769216-dd32-4edc-8e73-b4873caa041d\") " pod="openshift-network-operator/iptables-alerter-c9vlh"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505648    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1ceff922-50ab-4e0f-b113-03719eb40bc9-cnibin\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505661    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8161e358-9105-4d99-88c1-98c17db09674-registration-dir\") pod \"aws-ebs-csi-driver-node-tj7ch\" (UID: \"8161e358-9105-4d99-88c1-98c17db09674\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tj7ch"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505670    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-sys\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505674    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/725d2cce-2a75-426f-8522-d1ddfbabe1b6-kubelet-config\") pod \"global-pull-secret-syncer-z7mhx\" (UID: \"725d2cce-2a75-426f-8522-d1ddfbabe1b6\") " pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505670    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1ceff922-50ab-4e0f-b113-03719eb40bc9-cnibin\") pod \"multus-additional-cni-plugins-hcmmq\" (UID: \"1ceff922-50ab-4e0f-b113-03719eb40bc9\") " pod="openshift-multus/multus-additional-cni-plugins-hcmmq"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505714    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/725d2cce-2a75-426f-8522-d1ddfbabe1b6-kubelet-config\") pod \"global-pull-secret-syncer-z7mhx\" (UID: \"725d2cce-2a75-426f-8522-d1ddfbabe1b6\") " pod="kube-system/global-pull-secret-syncer-z7mhx"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505699    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-systemd-units\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505743    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-system-cni-dir\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505767    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-run-netns\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505794    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505821    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-lib-modules\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505837    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505840    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-system-cni-dir\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.505797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505823    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-run-netns\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.507750 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505796    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-systemd-units\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.507750 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505843    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-cnibin\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.507750 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505918    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-run-multus-certs\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.507750 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505928    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-lib-modules\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.507750 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505931    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-cnibin\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.507750 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505941    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-etc-kubernetes\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.507750 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505978    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f1622f2e-319b-4259-b314-a4178b7875f0-root\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:29.507750 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505977    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-host-run-multus-certs\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.507750 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505995    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1fda3255-2b22-40a0-870b-7b940977e197-etc-kubernetes\") pod \"multus-sdlcz\" (UID: \"1fda3255-2b22-40a0-870b-7b940977e197\") " pod="openshift-multus/multus-sdlcz"
Oct 30 19:21:29.507750 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.505997    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba4ee546-c9c1-4573-b11f-6ba0621972d1-host\") pod \"node-ca-tgt5g\" (UID: \"ba4ee546-c9c1-4573-b11f-6ba0621972d1\") " pod="openshift-image-registry/node-ca-tgt5g"
Oct 30 19:21:29.507750 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.506016    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba4ee546-c9c1-4573-b11f-6ba0621972d1-host\") pod \"node-ca-tgt5g\" (UID: \"ba4ee546-c9c1-4573-b11f-6ba0621972d1\") " pod="openshift-image-registry/node-ca-tgt5g"
Oct 30 19:21:29.507750 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.506026    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-etc-openvswitch\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.507750 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.506017    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f1622f2e-319b-4259-b314-a4178b7875f0-root\") pod \"node-exporter-jqs6k\" (UID: \"f1622f2e-319b-4259-b314-a4178b7875f0\") " pod="openshift-monitoring/node-exporter-jqs6k"
Oct 30 19:21:29.507750 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.506051    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-run-ovn-kubernetes\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.507750 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.506067    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-run\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.507750 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.506117    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-etc-openvswitch\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:29.507750 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.506130    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a8635205-1f61-4a17-bc1f-7876c5536f9c-run\") pod \"tuned-dvzwm\" (UID: \"a8635205-1f61-4a17-bc1f-7876c5536f9c\") " pod="openshift-cluster-node-tuning-operator/tuned-dvzwm"
Oct 30 19:21:29.507750 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:29.506130    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23ec668e-b2fb-471f-a7c7-13095a3dcf7f-host-run-ovn-kubernetes\") pod \"ovnkube-node-8pv94\" (UID: \"23ec668e-b2fb-471f-a7c7-13095a3dcf7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:30.530783 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:30.530750    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vn5fp"
Oct 30 19:21:31.810180 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:31.810151    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-66d96d76c5-nmsds"
Oct 30 19:21:32.866695 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:32.866656    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:32.885792 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:32.885758    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:33.127332 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:33.127239    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:33.706543 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:33.706505    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:21:35.697792 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:35.697756    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:21:36.846038 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:36.846004    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-mffps"
Oct 30 19:21:39.389489 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:39.389405    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:39.389843 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:39.389658    5907 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 19:21:42.733732 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:42.733698    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j"
Oct 30 19:21:47.498993 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:21:47.498951    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:21:48.067962 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:48.067930    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 19:21:48.068308 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:48.068271    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="40121914-4632-47b7-bc42-8299b3d6d6bd" containerName="alertmanager" containerID="cri-o://ef67942c95d5a202367f431a4db2814cf5e0648e6fd022bdb67f1d4dca39abd3" gracePeriod=120
Oct 30 19:21:48.068403 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:48.068302    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="40121914-4632-47b7-bc42-8299b3d6d6bd" containerName="kube-rbac-proxy-metric" containerID="cri-o://e2099a798a3a67494fbe9d71f71a4e97f9e4af74aec44ef1cab6314ef040ec4b" gracePeriod=120
Oct 30 19:21:48.068463 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:48.068369    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="40121914-4632-47b7-bc42-8299b3d6d6bd" containerName="prom-label-proxy" containerID="cri-o://3be76fada0592d95f1f110abdeb2e3408a0c3e56ef11b553bbba7d7e977eefe1" gracePeriod=120
Oct 30 19:21:48.068463 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:48.068428    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="40121914-4632-47b7-bc42-8299b3d6d6bd" containerName="kube-rbac-proxy" containerID="cri-o://c186ea0bc6198666b986912896187eb7190024431a7e9c87b41837b675f2a668" gracePeriod=120
Oct 30 19:21:48.068571 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:48.068372    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="40121914-4632-47b7-bc42-8299b3d6d6bd" containerName="config-reloader" containerID="cri-o://cbab46292c95c2c7cc8ac5d2214a7463e84a46a9a7fb1fe8698069253353dc88" gracePeriod=120
Oct 30 19:21:48.068571 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:48.068376    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="40121914-4632-47b7-bc42-8299b3d6d6bd" containerName="kube-rbac-proxy-web" containerID="cri-o://49fce01f5692865174f6bd212e20c9f5afd439845a6593959cf5520dbf2fdd5d" gracePeriod=120
Oct 30 19:21:48.446241 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:48.446207    5907 generic.go:356] "Generic (PLEG): container finished" podID="40121914-4632-47b7-bc42-8299b3d6d6bd" containerID="3be76fada0592d95f1f110abdeb2e3408a0c3e56ef11b553bbba7d7e977eefe1" exitCode=0
Oct 30 19:21:48.446241 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:48.446235    5907 generic.go:356] "Generic (PLEG): container finished" podID="40121914-4632-47b7-bc42-8299b3d6d6bd" containerID="e2099a798a3a67494fbe9d71f71a4e97f9e4af74aec44ef1cab6314ef040ec4b" exitCode=0
Oct 30 19:21:48.446241 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:48.446241    5907 generic.go:356] "Generic (PLEG): container finished" podID="40121914-4632-47b7-bc42-8299b3d6d6bd" containerID="c186ea0bc6198666b986912896187eb7190024431a7e9c87b41837b675f2a668" exitCode=0
Oct 30 19:21:48.446241 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:48.446247    5907 generic.go:356] "Generic (PLEG): container finished" podID="40121914-4632-47b7-bc42-8299b3d6d6bd" containerID="cbab46292c95c2c7cc8ac5d2214a7463e84a46a9a7fb1fe8698069253353dc88" exitCode=0
Oct 30 19:21:48.446506 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:48.446254    5907 generic.go:356] "Generic (PLEG): container finished" podID="40121914-4632-47b7-bc42-8299b3d6d6bd" containerID="ef67942c95d5a202367f431a4db2814cf5e0648e6fd022bdb67f1d4dca39abd3" exitCode=0
Oct 30 19:21:48.446506 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:48.446284    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"40121914-4632-47b7-bc42-8299b3d6d6bd","Type":"ContainerDied","Data":"3be76fada0592d95f1f110abdeb2e3408a0c3e56ef11b553bbba7d7e977eefe1"}
Oct 30 19:21:48.446506 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:48.446354    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"40121914-4632-47b7-bc42-8299b3d6d6bd","Type":"ContainerDied","Data":"e2099a798a3a67494fbe9d71f71a4e97f9e4af74aec44ef1cab6314ef040ec4b"}
Oct 30 19:21:48.446506 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:48.446384    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"40121914-4632-47b7-bc42-8299b3d6d6bd","Type":"ContainerDied","Data":"c186ea0bc6198666b986912896187eb7190024431a7e9c87b41837b675f2a668"}
Oct 30 19:21:48.446506 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:48.446408    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"40121914-4632-47b7-bc42-8299b3d6d6bd","Type":"ContainerDied","Data":"cbab46292c95c2c7cc8ac5d2214a7463e84a46a9a7fb1fe8698069253353dc88"}
Oct 30 19:21:48.446506 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:48.446432    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"40121914-4632-47b7-bc42-8299b3d6d6bd","Type":"ContainerDied","Data":"ef67942c95d5a202367f431a4db2814cf5e0648e6fd022bdb67f1d4dca39abd3"}
Oct 30 19:21:49.345521 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.345487    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.380705 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.380671    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-config-volume\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-config-volume\") pod \"40121914-4632-47b7-bc42-8299b3d6d6bd\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") "
Oct 30 19:21:49.380908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.380729    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"40121914-4632-47b7-bc42-8299b3d6d6bd\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") "
Oct 30 19:21:49.380908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.380762    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/40121914-4632-47b7-bc42-8299b3d6d6bd-kube-api-access-6vlf5\" (UniqueName: \"kubernetes.io/projected/40121914-4632-47b7-bc42-8299b3d6d6bd-kube-api-access-6vlf5\") pod \"40121914-4632-47b7-bc42-8299b3d6d6bd\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") "
Oct 30 19:21:49.380908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.380821    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40121914-4632-47b7-bc42-8299b3d6d6bd-config-out\" (UniqueName: \"kubernetes.io/empty-dir/40121914-4632-47b7-bc42-8299b3d6d6bd-config-out\") pod \"40121914-4632-47b7-bc42-8299b3d6d6bd\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") "
Oct 30 19:21:49.381074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.380910    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-kube-rbac-proxy-web\") pod \"40121914-4632-47b7-bc42-8299b3d6d6bd\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") "
Oct 30 19:21:49.381074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.380952    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40121914-4632-47b7-bc42-8299b3d6d6bd-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/40121914-4632-47b7-bc42-8299b3d6d6bd-alertmanager-main-db\") pod \"40121914-4632-47b7-bc42-8299b3d6d6bd\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") "
Oct 30 19:21:49.381074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.380989    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/40121914-4632-47b7-bc42-8299b3d6d6bd-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40121914-4632-47b7-bc42-8299b3d6d6bd-metrics-client-ca\") pod \"40121914-4632-47b7-bc42-8299b3d6d6bd\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") "
Oct 30 19:21:49.381074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.381046    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/40121914-4632-47b7-bc42-8299b3d6d6bd-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40121914-4632-47b7-bc42-8299b3d6d6bd-alertmanager-trusted-ca-bundle\") pod \"40121914-4632-47b7-bc42-8299b3d6d6bd\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") "
Oct 30 19:21:49.381274 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.381090    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-kube-rbac-proxy\") pod \"40121914-4632-47b7-bc42-8299b3d6d6bd\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") "
Oct 30 19:21:49.382305 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.381855    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40121914-4632-47b7-bc42-8299b3d6d6bd-alertmanager-main-db" pod "40121914-4632-47b7-bc42-8299b3d6d6bd" (UID: "40121914-4632-47b7-bc42-8299b3d6d6bd"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:21:49.382412 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.382161    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40121914-4632-47b7-bc42-8299b3d6d6bd-alertmanager-trusted-ca-bundle" pod "40121914-4632-47b7-bc42-8299b3d6d6bd" (UID: "40121914-4632-47b7-bc42-8299b3d6d6bd"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:21:49.382513 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.382492    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-web-config\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-web-config\") pod \"40121914-4632-47b7-bc42-8299b3d6d6bd\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") "
Oct 30 19:21:49.383324 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.383234    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40121914-4632-47b7-bc42-8299b3d6d6bd-metrics-client-ca" pod "40121914-4632-47b7-bc42-8299b3d6d6bd" (UID: "40121914-4632-47b7-bc42-8299b3d6d6bd"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:21:49.383411 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.382559    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-main-tls\") pod \"40121914-4632-47b7-bc42-8299b3d6d6bd\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") "
Oct 30 19:21:49.383411 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.383393    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/40121914-4632-47b7-bc42-8299b3d6d6bd-tls-assets\" (UniqueName: \"kubernetes.io/projected/40121914-4632-47b7-bc42-8299b3d6d6bd-tls-assets\") pod \"40121914-4632-47b7-bc42-8299b3d6d6bd\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") "
Oct 30 19:21:49.383564 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.383429    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-cluster-tls-config\") pod \"40121914-4632-47b7-bc42-8299b3d6d6bd\" (UID: \"40121914-4632-47b7-bc42-8299b3d6d6bd\") "
Oct 30 19:21:49.383931 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.383893    5907 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/40121914-4632-47b7-bc42-8299b3d6d6bd-alertmanager-main-db\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:49.383931 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.383924    5907 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40121914-4632-47b7-bc42-8299b3d6d6bd-metrics-client-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:49.384061 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.383949    5907 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40121914-4632-47b7-bc42-8299b3d6d6bd-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:49.384238 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.384210    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40121914-4632-47b7-bc42-8299b3d6d6bd-config-out" pod "40121914-4632-47b7-bc42-8299b3d6d6bd" (UID: "40121914-4632-47b7-bc42-8299b3d6d6bd"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:21:49.385273 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.384533    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-config-volume" pod "40121914-4632-47b7-bc42-8299b3d6d6bd" (UID: "40121914-4632-47b7-bc42-8299b3d6d6bd"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:21:49.385811 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.385451    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40121914-4632-47b7-bc42-8299b3d6d6bd-kube-api-access-6vlf5" pod "40121914-4632-47b7-bc42-8299b3d6d6bd" (UID: "40121914-4632-47b7-bc42-8299b3d6d6bd"). InnerVolumeSpecName "kube-api-access-6vlf5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:21:49.385811 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.385497    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-kube-rbac-proxy-metric" pod "40121914-4632-47b7-bc42-8299b3d6d6bd" (UID: "40121914-4632-47b7-bc42-8299b3d6d6bd"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:21:49.387646 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.386399    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-kube-rbac-proxy-web" pod "40121914-4632-47b7-bc42-8299b3d6d6bd" (UID: "40121914-4632-47b7-bc42-8299b3d6d6bd"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:21:49.392281 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.392206    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40121914-4632-47b7-bc42-8299b3d6d6bd-tls-assets" pod "40121914-4632-47b7-bc42-8299b3d6d6bd" (UID: "40121914-4632-47b7-bc42-8299b3d6d6bd"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:21:49.392281 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.392217    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-main-tls" pod "40121914-4632-47b7-bc42-8299b3d6d6bd" (UID: "40121914-4632-47b7-bc42-8299b3d6d6bd"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:21:49.393499 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.393470    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-kube-rbac-proxy" pod "40121914-4632-47b7-bc42-8299b3d6d6bd" (UID: "40121914-4632-47b7-bc42-8299b3d6d6bd"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:21:49.394688 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.394662    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-cluster-tls-config" pod "40121914-4632-47b7-bc42-8299b3d6d6bd" (UID: "40121914-4632-47b7-bc42-8299b3d6d6bd"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:21:49.401939 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.401914    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-web-config" pod "40121914-4632-47b7-bc42-8299b3d6d6bd" (UID: "40121914-4632-47b7-bc42-8299b3d6d6bd"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:21:49.453818 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.453776    5907 generic.go:356] "Generic (PLEG): container finished" podID="40121914-4632-47b7-bc42-8299b3d6d6bd" containerID="49fce01f5692865174f6bd212e20c9f5afd439845a6593959cf5520dbf2fdd5d" exitCode=0
Oct 30 19:21:49.454024 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.453834    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"40121914-4632-47b7-bc42-8299b3d6d6bd","Type":"ContainerDied","Data":"49fce01f5692865174f6bd212e20c9f5afd439845a6593959cf5520dbf2fdd5d"}
Oct 30 19:21:49.454024 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.453891    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.454024 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.453980    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"40121914-4632-47b7-bc42-8299b3d6d6bd","Type":"ContainerDied","Data":"990f36d93789ddcfa3d80eae721de26344c8e54c43783b60e92e3128c3a8d2ae"}
Oct 30 19:21:49.454194 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.454034    5907 scope.go:122] "RemoveContainer" containerID="3be76fada0592d95f1f110abdeb2e3408a0c3e56ef11b553bbba7d7e977eefe1"
Oct 30 19:21:49.463668 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.463646    5907 scope.go:122] "RemoveContainer" containerID="e2099a798a3a67494fbe9d71f71a4e97f9e4af74aec44ef1cab6314ef040ec4b"
Oct 30 19:21:49.475289 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.475268    5907 scope.go:122] "RemoveContainer" containerID="c186ea0bc6198666b986912896187eb7190024431a7e9c87b41837b675f2a668"
Oct 30 19:21:49.482769 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.482739    5907 scope.go:122] "RemoveContainer" containerID="49fce01f5692865174f6bd212e20c9f5afd439845a6593959cf5520dbf2fdd5d"
Oct 30 19:21:49.485035 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.485016    5907 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:49.485135 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.485040    5907 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-web-config\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:49.485135 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.485057    5907 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-main-tls\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:49.485135 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.485072    5907 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/40121914-4632-47b7-bc42-8299b3d6d6bd-tls-assets\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:49.485135 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.485085    5907 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-cluster-tls-config\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:49.485135 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.485099    5907 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:49.485135 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.485113    5907 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:49.485135 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.485130    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vlf5\" (UniqueName: \"kubernetes.io/projected/40121914-4632-47b7-bc42-8299b3d6d6bd-kube-api-access-6vlf5\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:49.485363 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.485145    5907 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/40121914-4632-47b7-bc42-8299b3d6d6bd-config-out\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:49.485363 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.485159    5907 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/40121914-4632-47b7-bc42-8299b3d6d6bd-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:49.490445 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.490428    5907 scope.go:122] "RemoveContainer" containerID="cbab46292c95c2c7cc8ac5d2214a7463e84a46a9a7fb1fe8698069253353dc88"
Oct 30 19:21:49.497531 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.497502    5907 scope.go:122] "RemoveContainer" containerID="ef67942c95d5a202367f431a4db2814cf5e0648e6fd022bdb67f1d4dca39abd3"
Oct 30 19:21:49.504994 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.504981    5907 scope.go:122] "RemoveContainer" containerID="65ce9926fc3f76d0f605f9b84cd747beac60ac1ea3c9e9b2a10d37644104a416"
Oct 30 19:21:49.511291 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.511268    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 19:21:49.512595 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.512554    5907 scope.go:122] "RemoveContainer" containerID="3be76fada0592d95f1f110abdeb2e3408a0c3e56ef11b553bbba7d7e977eefe1"
Oct 30 19:21:49.512893 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:21:49.512848    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3be76fada0592d95f1f110abdeb2e3408a0c3e56ef11b553bbba7d7e977eefe1\": container with ID starting with 3be76fada0592d95f1f110abdeb2e3408a0c3e56ef11b553bbba7d7e977eefe1 not found: ID does not exist" containerID="3be76fada0592d95f1f110abdeb2e3408a0c3e56ef11b553bbba7d7e977eefe1"
Oct 30 19:21:49.512937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.512904    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3be76fada0592d95f1f110abdeb2e3408a0c3e56ef11b553bbba7d7e977eefe1"} err="failed to get container status \"3be76fada0592d95f1f110abdeb2e3408a0c3e56ef11b553bbba7d7e977eefe1\": rpc error: code = NotFound desc = could not find container \"3be76fada0592d95f1f110abdeb2e3408a0c3e56ef11b553bbba7d7e977eefe1\": container with ID starting with 3be76fada0592d95f1f110abdeb2e3408a0c3e56ef11b553bbba7d7e977eefe1 not found: ID does not exist"
Oct 30 19:21:49.512990 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.512939    5907 scope.go:122] "RemoveContainer" containerID="e2099a798a3a67494fbe9d71f71a4e97f9e4af74aec44ef1cab6314ef040ec4b"
Oct 30 19:21:49.513145 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:21:49.513130    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2099a798a3a67494fbe9d71f71a4e97f9e4af74aec44ef1cab6314ef040ec4b\": container with ID starting with e2099a798a3a67494fbe9d71f71a4e97f9e4af74aec44ef1cab6314ef040ec4b not found: ID does not exist" containerID="e2099a798a3a67494fbe9d71f71a4e97f9e4af74aec44ef1cab6314ef040ec4b"
Oct 30 19:21:49.513200 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.513148    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2099a798a3a67494fbe9d71f71a4e97f9e4af74aec44ef1cab6314ef040ec4b"} err="failed to get container status \"e2099a798a3a67494fbe9d71f71a4e97f9e4af74aec44ef1cab6314ef040ec4b\": rpc error: code = NotFound desc = could not find container \"e2099a798a3a67494fbe9d71f71a4e97f9e4af74aec44ef1cab6314ef040ec4b\": container with ID starting with e2099a798a3a67494fbe9d71f71a4e97f9e4af74aec44ef1cab6314ef040ec4b not found: ID does not exist"
Oct 30 19:21:49.513200 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.513160    5907 scope.go:122] "RemoveContainer" containerID="c186ea0bc6198666b986912896187eb7190024431a7e9c87b41837b675f2a668"
Oct 30 19:21:49.513396 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:21:49.513380    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c186ea0bc6198666b986912896187eb7190024431a7e9c87b41837b675f2a668\": container with ID starting with c186ea0bc6198666b986912896187eb7190024431a7e9c87b41837b675f2a668 not found: ID does not exist" containerID="c186ea0bc6198666b986912896187eb7190024431a7e9c87b41837b675f2a668"
Oct 30 19:21:49.513442 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.513403    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c186ea0bc6198666b986912896187eb7190024431a7e9c87b41837b675f2a668"} err="failed to get container status \"c186ea0bc6198666b986912896187eb7190024431a7e9c87b41837b675f2a668\": rpc error: code = NotFound desc = could not find container \"c186ea0bc6198666b986912896187eb7190024431a7e9c87b41837b675f2a668\": container with ID starting with c186ea0bc6198666b986912896187eb7190024431a7e9c87b41837b675f2a668 not found: ID does not exist"
Oct 30 19:21:49.513442 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.513418    5907 scope.go:122] "RemoveContainer" containerID="49fce01f5692865174f6bd212e20c9f5afd439845a6593959cf5520dbf2fdd5d"
Oct 30 19:21:49.513677 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:21:49.513662    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49fce01f5692865174f6bd212e20c9f5afd439845a6593959cf5520dbf2fdd5d\": container with ID starting with 49fce01f5692865174f6bd212e20c9f5afd439845a6593959cf5520dbf2fdd5d not found: ID does not exist" containerID="49fce01f5692865174f6bd212e20c9f5afd439845a6593959cf5520dbf2fdd5d"
Oct 30 19:21:49.513722 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.513681    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49fce01f5692865174f6bd212e20c9f5afd439845a6593959cf5520dbf2fdd5d"} err="failed to get container status \"49fce01f5692865174f6bd212e20c9f5afd439845a6593959cf5520dbf2fdd5d\": rpc error: code = NotFound desc = could not find container \"49fce01f5692865174f6bd212e20c9f5afd439845a6593959cf5520dbf2fdd5d\": container with ID starting with 49fce01f5692865174f6bd212e20c9f5afd439845a6593959cf5520dbf2fdd5d not found: ID does not exist"
Oct 30 19:21:49.513722 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.513693    5907 scope.go:122] "RemoveContainer" containerID="cbab46292c95c2c7cc8ac5d2214a7463e84a46a9a7fb1fe8698069253353dc88"
Oct 30 19:21:49.513872 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:21:49.513848    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cbab46292c95c2c7cc8ac5d2214a7463e84a46a9a7fb1fe8698069253353dc88\": container with ID starting with cbab46292c95c2c7cc8ac5d2214a7463e84a46a9a7fb1fe8698069253353dc88 not found: ID does not exist" containerID="cbab46292c95c2c7cc8ac5d2214a7463e84a46a9a7fb1fe8698069253353dc88"
Oct 30 19:21:49.513917 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.513888    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbab46292c95c2c7cc8ac5d2214a7463e84a46a9a7fb1fe8698069253353dc88"} err="failed to get container status \"cbab46292c95c2c7cc8ac5d2214a7463e84a46a9a7fb1fe8698069253353dc88\": rpc error: code = NotFound desc = could not find container \"cbab46292c95c2c7cc8ac5d2214a7463e84a46a9a7fb1fe8698069253353dc88\": container with ID starting with cbab46292c95c2c7cc8ac5d2214a7463e84a46a9a7fb1fe8698069253353dc88 not found: ID does not exist"
Oct 30 19:21:49.513917 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.513899    5907 scope.go:122] "RemoveContainer" containerID="ef67942c95d5a202367f431a4db2814cf5e0648e6fd022bdb67f1d4dca39abd3"
Oct 30 19:21:49.514101 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:21:49.514083    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef67942c95d5a202367f431a4db2814cf5e0648e6fd022bdb67f1d4dca39abd3\": container with ID starting with ef67942c95d5a202367f431a4db2814cf5e0648e6fd022bdb67f1d4dca39abd3 not found: ID does not exist" containerID="ef67942c95d5a202367f431a4db2814cf5e0648e6fd022bdb67f1d4dca39abd3"
Oct 30 19:21:49.514144 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.514106    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef67942c95d5a202367f431a4db2814cf5e0648e6fd022bdb67f1d4dca39abd3"} err="failed to get container status \"ef67942c95d5a202367f431a4db2814cf5e0648e6fd022bdb67f1d4dca39abd3\": rpc error: code = NotFound desc = could not find container \"ef67942c95d5a202367f431a4db2814cf5e0648e6fd022bdb67f1d4dca39abd3\": container with ID starting with ef67942c95d5a202367f431a4db2814cf5e0648e6fd022bdb67f1d4dca39abd3 not found: ID does not exist"
Oct 30 19:21:49.514144 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.514120    5907 scope.go:122] "RemoveContainer" containerID="65ce9926fc3f76d0f605f9b84cd747beac60ac1ea3c9e9b2a10d37644104a416"
Oct 30 19:21:49.514378 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:21:49.514360    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65ce9926fc3f76d0f605f9b84cd747beac60ac1ea3c9e9b2a10d37644104a416\": container with ID starting with 65ce9926fc3f76d0f605f9b84cd747beac60ac1ea3c9e9b2a10d37644104a416 not found: ID does not exist" containerID="65ce9926fc3f76d0f605f9b84cd747beac60ac1ea3c9e9b2a10d37644104a416"
Oct 30 19:21:49.514473 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.514393    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65ce9926fc3f76d0f605f9b84cd747beac60ac1ea3c9e9b2a10d37644104a416"} err="failed to get container status \"65ce9926fc3f76d0f605f9b84cd747beac60ac1ea3c9e9b2a10d37644104a416\": rpc error: code = NotFound desc = could not find container \"65ce9926fc3f76d0f605f9b84cd747beac60ac1ea3c9e9b2a10d37644104a416\": container with ID starting with 65ce9926fc3f76d0f605f9b84cd747beac60ac1ea3c9e9b2a10d37644104a416 not found: ID does not exist"
Oct 30 19:21:49.517672 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.517651    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 19:21:49.542595 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.542561    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 19:21:49.542929 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.542913    5907 cpu_manager.go:414] "containerMap: removing container" podUID="40121914-4632-47b7-bc42-8299b3d6d6bd" containerName="kube-rbac-proxy-metric"
Oct 30 19:21:49.543004 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.542931    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40121914-4632-47b7-bc42-8299b3d6d6bd" containerName="kube-rbac-proxy-metric"
Oct 30 19:21:49.543004 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.542942    5907 cpu_manager.go:414] "containerMap: removing container" podUID="40121914-4632-47b7-bc42-8299b3d6d6bd" containerName="init-config-reloader"
Oct 30 19:21:49.543004 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.542947    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40121914-4632-47b7-bc42-8299b3d6d6bd" containerName="init-config-reloader"
Oct 30 19:21:49.543004 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.542957    5907 cpu_manager.go:414] "containerMap: removing container" podUID="40121914-4632-47b7-bc42-8299b3d6d6bd" containerName="kube-rbac-proxy-web"
Oct 30 19:21:49.543004 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.542961    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40121914-4632-47b7-bc42-8299b3d6d6bd" containerName="kube-rbac-proxy-web"
Oct 30 19:21:49.543004 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.542986    5907 cpu_manager.go:414] "containerMap: removing container" podUID="40121914-4632-47b7-bc42-8299b3d6d6bd" containerName="prom-label-proxy"
Oct 30 19:21:49.543004 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.542990    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40121914-4632-47b7-bc42-8299b3d6d6bd" containerName="prom-label-proxy"
Oct 30 19:21:49.543004 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.542999    5907 cpu_manager.go:414] "containerMap: removing container" podUID="40121914-4632-47b7-bc42-8299b3d6d6bd" containerName="config-reloader"
Oct 30 19:21:49.543004 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.543003    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40121914-4632-47b7-bc42-8299b3d6d6bd" containerName="config-reloader"
Oct 30 19:21:49.543261 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.543012    5907 cpu_manager.go:414] "containerMap: removing container" podUID="40121914-4632-47b7-bc42-8299b3d6d6bd" containerName="alertmanager"
Oct 30 19:21:49.543261 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.543017    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40121914-4632-47b7-bc42-8299b3d6d6bd" containerName="alertmanager"
Oct 30 19:21:49.543261 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.543025    5907 cpu_manager.go:414] "containerMap: removing container" podUID="40121914-4632-47b7-bc42-8299b3d6d6bd" containerName="kube-rbac-proxy"
Oct 30 19:21:49.543261 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.543032    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40121914-4632-47b7-bc42-8299b3d6d6bd" containerName="kube-rbac-proxy"
Oct 30 19:21:49.543261 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.543084    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="40121914-4632-47b7-bc42-8299b3d6d6bd" containerName="prom-label-proxy"
Oct 30 19:21:49.543261 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.543093    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="40121914-4632-47b7-bc42-8299b3d6d6bd" containerName="config-reloader"
Oct 30 19:21:49.543261 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.543100    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="40121914-4632-47b7-bc42-8299b3d6d6bd" containerName="alertmanager"
Oct 30 19:21:49.543261 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.543106    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="40121914-4632-47b7-bc42-8299b3d6d6bd" containerName="kube-rbac-proxy"
Oct 30 19:21:49.543261 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.543111    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="40121914-4632-47b7-bc42-8299b3d6d6bd" containerName="kube-rbac-proxy-metric"
Oct 30 19:21:49.543261 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.543117    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="40121914-4632-47b7-bc42-8299b3d6d6bd" containerName="init-config-reloader"
Oct 30 19:21:49.543261 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.543122    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="40121914-4632-47b7-bc42-8299b3d6d6bd" containerName="kube-rbac-proxy-web"
Oct 30 19:21:49.564530 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.564509    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.574504 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.574480    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 19:21:49.581454 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.581424    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 19:21:49.581592 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.581468    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 19:21:49.581592 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.581478    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 19:21:49.581592 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.581459    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 19:21:49.581592 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.581498    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 19:21:49.581749 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.581615    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-qh4dk\""
Oct 30 19:21:49.581749 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.581673    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 19:21:49.581749 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.581676    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 19:21:49.581924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.581908    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 19:21:49.586284 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.586266    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/70c6b30c-9245-4fd7-908b-9fc3eef24d91-web-config\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.586378 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.586301    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/70c6b30c-9245-4fd7-908b-9fc3eef24d91-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.586378 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.586335    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/70c6b30c-9245-4fd7-908b-9fc3eef24d91-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.586378 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.586352    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/70c6b30c-9245-4fd7-908b-9fc3eef24d91-tls-assets\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.586538 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.586507    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/70c6b30c-9245-4fd7-908b-9fc3eef24d91-config-out\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.586538 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.586529    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/70c6b30c-9245-4fd7-908b-9fc3eef24d91-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.586600 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.586557    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/70c6b30c-9245-4fd7-908b-9fc3eef24d91-config-volume\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.586653 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.586609    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/70c6b30c-9245-4fd7-908b-9fc3eef24d91-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.586730 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.586711    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70c6b30c-9245-4fd7-908b-9fc3eef24d91-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.586839 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.586821    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpbx6\" (UniqueName: \"kubernetes.io/projected/70c6b30c-9245-4fd7-908b-9fc3eef24d91-kube-api-access-kpbx6\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.586933 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.586914    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/70c6b30c-9245-4fd7-908b-9fc3eef24d91-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.586975 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.586948    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/70c6b30c-9245-4fd7-908b-9fc3eef24d91-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.587011 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.586980    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70c6b30c-9245-4fd7-908b-9fc3eef24d91-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.594856 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.594839    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 19:21:49.688241 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.688154    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/70c6b30c-9245-4fd7-908b-9fc3eef24d91-web-config\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.688241 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.688197    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/70c6b30c-9245-4fd7-908b-9fc3eef24d91-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.688241 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.688230    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/70c6b30c-9245-4fd7-908b-9fc3eef24d91-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.688523 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.688247    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/70c6b30c-9245-4fd7-908b-9fc3eef24d91-tls-assets\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.688523 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.688268    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/70c6b30c-9245-4fd7-908b-9fc3eef24d91-config-out\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.688523 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.688281    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/70c6b30c-9245-4fd7-908b-9fc3eef24d91-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.688523 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.688305    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/70c6b30c-9245-4fd7-908b-9fc3eef24d91-config-volume\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.688523 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.688329    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/70c6b30c-9245-4fd7-908b-9fc3eef24d91-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.688523 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.688364    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70c6b30c-9245-4fd7-908b-9fc3eef24d91-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.688523 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.688388    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpbx6\" (UniqueName: \"kubernetes.io/projected/70c6b30c-9245-4fd7-908b-9fc3eef24d91-kube-api-access-kpbx6\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.688523 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.688418    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/70c6b30c-9245-4fd7-908b-9fc3eef24d91-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.688523 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.688443    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/70c6b30c-9245-4fd7-908b-9fc3eef24d91-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.688523 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.688465    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70c6b30c-9245-4fd7-908b-9fc3eef24d91-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.689069 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.688582    5907 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 19:21:49.689069 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.688848    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/70c6b30c-9245-4fd7-908b-9fc3eef24d91-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.689972 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.689944    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70c6b30c-9245-4fd7-908b-9fc3eef24d91-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.690097 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.689969    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70c6b30c-9245-4fd7-908b-9fc3eef24d91-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.691763 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.691711    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/70c6b30c-9245-4fd7-908b-9fc3eef24d91-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.691763 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.691727    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/70c6b30c-9245-4fd7-908b-9fc3eef24d91-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.691956 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.691939    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/70c6b30c-9245-4fd7-908b-9fc3eef24d91-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.692101 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.692079    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/70c6b30c-9245-4fd7-908b-9fc3eef24d91-config-out\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.692181 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.692135    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/70c6b30c-9245-4fd7-908b-9fc3eef24d91-web-config\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.692374 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.692359    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/70c6b30c-9245-4fd7-908b-9fc3eef24d91-config-volume\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.692726 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.692706    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/70c6b30c-9245-4fd7-908b-9fc3eef24d91-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.693200 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.693178    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/70c6b30c-9245-4fd7-908b-9fc3eef24d91-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.693551 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.693533    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/70c6b30c-9245-4fd7-908b-9fc3eef24d91-tls-assets\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.710450 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.710428    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpbx6\" (UniqueName: \"kubernetes.io/projected/70c6b30c-9245-4fd7-908b-9fc3eef24d91-kube-api-access-kpbx6\") pod \"alertmanager-main-0\" (UID: \"70c6b30c-9245-4fd7-908b-9fc3eef24d91\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:49.957259 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:49.957171    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 19:21:50.231488 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:50.231424    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8pv94"
Oct 30 19:21:50.285872 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:50.285825    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="40121914-4632-47b7-bc42-8299b3d6d6bd" path="/var/lib/kubelet/pods/40121914-4632-47b7-bc42-8299b3d6d6bd/volumes"
Oct 30 19:21:50.319219 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:50.319178    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 19:21:50.335633 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:50.335603    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70c6b30c_9245_4fd7_908b_9fc3eef24d91.slice/crio-3bf8a54510101686ef4793672dedf0e4ec9c3025d1b313eed337ba4081bcc746 WatchSource:0}: Error finding container 3bf8a54510101686ef4793672dedf0e4ec9c3025d1b313eed337ba4081bcc746: Status 404 returned error can't find the container with id 3bf8a54510101686ef4793672dedf0e4ec9c3025d1b313eed337ba4081bcc746
Oct 30 19:21:50.459665 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:50.459634    5907 generic.go:356] "Generic (PLEG): container finished" podID="70c6b30c-9245-4fd7-908b-9fc3eef24d91" containerID="a07532d8803ec1c792dfd2b4195c345f46895c21bbe50d439c0c2bebc11c5d03" exitCode=0
Oct 30 19:21:50.460124 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:50.459715    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"70c6b30c-9245-4fd7-908b-9fc3eef24d91","Type":"ContainerDied","Data":"a07532d8803ec1c792dfd2b4195c345f46895c21bbe50d439c0c2bebc11c5d03"}
Oct 30 19:21:50.460124 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:50.459778    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"70c6b30c-9245-4fd7-908b-9fc3eef24d91","Type":"ContainerStarted","Data":"3bf8a54510101686ef4793672dedf0e4ec9c3025d1b313eed337ba4081bcc746"}
Oct 30 19:21:51.468303 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:51.468260    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"70c6b30c-9245-4fd7-908b-9fc3eef24d91","Type":"ContainerStarted","Data":"9e6ebb9462e3b37db72af822018feb4b654ff0fb2c3582eb1aa884045adcfbb9"}
Oct 30 19:21:51.468657 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:51.468376    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"70c6b30c-9245-4fd7-908b-9fc3eef24d91","Type":"ContainerStarted","Data":"cfb9f0b902dcceef727100af083b23fb3d2f069e54af3aa72b5f0ede8177acb7"}
Oct 30 19:21:51.468657 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:51.468405    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"70c6b30c-9245-4fd7-908b-9fc3eef24d91","Type":"ContainerStarted","Data":"73e03265ffb662d1166b909b39208ae40763ba8f979958846eec597ecd6bbeda"}
Oct 30 19:21:51.468657 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:51.468438    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"70c6b30c-9245-4fd7-908b-9fc3eef24d91","Type":"ContainerStarted","Data":"d2b4a89c932a4de323f3f4b49a2a83af32d0881ee79b286ab4a2569437cb595d"}
Oct 30 19:21:51.468657 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:51.468474    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"70c6b30c-9245-4fd7-908b-9fc3eef24d91","Type":"ContainerStarted","Data":"8d39a86eb887db22d0b298fbef410cc6c04a11f305d7cb292ade5be5b05ce480"}
Oct 30 19:21:51.468657 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:51.468510    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"70c6b30c-9245-4fd7-908b-9fc3eef24d91","Type":"ContainerStarted","Data":"539be02b6386620b470e6ab1883de79c1584580ec8d9bb82ae501fd96ab7e8e7"}
Oct 30 19:21:51.533753 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:51.533703    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.521602671 podStartE2EDuration="2.533690521s" totalImagesPullingTime="0s" totalInitContainerRuntime="12.08785ms" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:51.532891466 +0000 UTC m=+23.375823405" watchObservedRunningTime="2026-10-30 19:21:51.533690521 +0000 UTC m=+23.376622460"
Oct 30 19:21:52.040591 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.040555    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-dbd6b58fb-r6mpp"]
Oct 30 19:21:52.062172 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.062140    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-dbd6b58fb-r6mpp"
Oct 30 19:21:52.086720 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.086687    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Oct 30 19:21:52.100278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.100234    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Oct 30 19:21:52.100278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.100262    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-7jzv4\""
Oct 30 19:21:52.100278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.100241    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Oct 30 19:21:52.100527 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.100313    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Oct 30 19:21:52.101066 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.101046    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-dbd6b58fb-r6mpp"]
Oct 30 19:21:52.115034 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.115004    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7c27d265-67e1-424e-85ff-38a8c30bd547-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-dbd6b58fb-r6mpp\" (UID: \"7c27d265-67e1-424e-85ff-38a8c30bd547\") " pod="openshift-monitoring/telemeter-client-dbd6b58fb-r6mpp"
Oct 30 19:21:52.115158 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.115126    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7c27d265-67e1-424e-85ff-38a8c30bd547-secret-telemeter-client\") pod \"telemeter-client-dbd6b58fb-r6mpp\" (UID: \"7c27d265-67e1-424e-85ff-38a8c30bd547\") " pod="openshift-monitoring/telemeter-client-dbd6b58fb-r6mpp"
Oct 30 19:21:52.115210 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.115159    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c27d265-67e1-424e-85ff-38a8c30bd547-serving-certs-ca-bundle\") pod \"telemeter-client-dbd6b58fb-r6mpp\" (UID: \"7c27d265-67e1-424e-85ff-38a8c30bd547\") " pod="openshift-monitoring/telemeter-client-dbd6b58fb-r6mpp"
Oct 30 19:21:52.115245 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.115230    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7c27d265-67e1-424e-85ff-38a8c30bd547-telemeter-client-tls\") pod \"telemeter-client-dbd6b58fb-r6mpp\" (UID: \"7c27d265-67e1-424e-85ff-38a8c30bd547\") " pod="openshift-monitoring/telemeter-client-dbd6b58fb-r6mpp"
Oct 30 19:21:52.115335 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.115319    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c27d265-67e1-424e-85ff-38a8c30bd547-telemeter-trusted-ca-bundle\") pod \"telemeter-client-dbd6b58fb-r6mpp\" (UID: \"7c27d265-67e1-424e-85ff-38a8c30bd547\") " pod="openshift-monitoring/telemeter-client-dbd6b58fb-r6mpp"
Oct 30 19:21:52.115367 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.115346    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4plf\" (UniqueName: \"kubernetes.io/projected/7c27d265-67e1-424e-85ff-38a8c30bd547-kube-api-access-b4plf\") pod \"telemeter-client-dbd6b58fb-r6mpp\" (UID: \"7c27d265-67e1-424e-85ff-38a8c30bd547\") " pod="openshift-monitoring/telemeter-client-dbd6b58fb-r6mpp"
Oct 30 19:21:52.115471 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.115456    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7c27d265-67e1-424e-85ff-38a8c30bd547-federate-client-tls\") pod \"telemeter-client-dbd6b58fb-r6mpp\" (UID: \"7c27d265-67e1-424e-85ff-38a8c30bd547\") " pod="openshift-monitoring/telemeter-client-dbd6b58fb-r6mpp"
Oct 30 19:21:52.115508 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.115490    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7c27d265-67e1-424e-85ff-38a8c30bd547-metrics-client-ca\") pod \"telemeter-client-dbd6b58fb-r6mpp\" (UID: \"7c27d265-67e1-424e-85ff-38a8c30bd547\") " pod="openshift-monitoring/telemeter-client-dbd6b58fb-r6mpp"
Oct 30 19:21:52.141153 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.141101    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Oct 30 19:21:52.155650 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.155619    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Oct 30 19:21:52.216102 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.216067    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7c27d265-67e1-424e-85ff-38a8c30bd547-secret-telemeter-client\") pod \"telemeter-client-dbd6b58fb-r6mpp\" (UID: \"7c27d265-67e1-424e-85ff-38a8c30bd547\") " pod="openshift-monitoring/telemeter-client-dbd6b58fb-r6mpp"
Oct 30 19:21:52.216102 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.216103    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c27d265-67e1-424e-85ff-38a8c30bd547-serving-certs-ca-bundle\") pod \"telemeter-client-dbd6b58fb-r6mpp\" (UID: \"7c27d265-67e1-424e-85ff-38a8c30bd547\") " pod="openshift-monitoring/telemeter-client-dbd6b58fb-r6mpp"
Oct 30 19:21:52.216308 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.216124    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7c27d265-67e1-424e-85ff-38a8c30bd547-telemeter-client-tls\") pod \"telemeter-client-dbd6b58fb-r6mpp\" (UID: \"7c27d265-67e1-424e-85ff-38a8c30bd547\") " pod="openshift-monitoring/telemeter-client-dbd6b58fb-r6mpp"
Oct 30 19:21:52.216308 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.216184    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c27d265-67e1-424e-85ff-38a8c30bd547-telemeter-trusted-ca-bundle\") pod \"telemeter-client-dbd6b58fb-r6mpp\" (UID: \"7c27d265-67e1-424e-85ff-38a8c30bd547\") " pod="openshift-monitoring/telemeter-client-dbd6b58fb-r6mpp"
Oct 30 19:21:52.216308 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.216212    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4plf\" (UniqueName: \"kubernetes.io/projected/7c27d265-67e1-424e-85ff-38a8c30bd547-kube-api-access-b4plf\") pod \"telemeter-client-dbd6b58fb-r6mpp\" (UID: \"7c27d265-67e1-424e-85ff-38a8c30bd547\") " pod="openshift-monitoring/telemeter-client-dbd6b58fb-r6mpp"
Oct 30 19:21:52.216308 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.216280    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7c27d265-67e1-424e-85ff-38a8c30bd547-federate-client-tls\") pod \"telemeter-client-dbd6b58fb-r6mpp\" (UID: \"7c27d265-67e1-424e-85ff-38a8c30bd547\") " pod="openshift-monitoring/telemeter-client-dbd6b58fb-r6mpp"
Oct 30 19:21:52.216488 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.216308    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7c27d265-67e1-424e-85ff-38a8c30bd547-metrics-client-ca\") pod \"telemeter-client-dbd6b58fb-r6mpp\" (UID: \"7c27d265-67e1-424e-85ff-38a8c30bd547\") " pod="openshift-monitoring/telemeter-client-dbd6b58fb-r6mpp"
Oct 30 19:21:52.216488 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.216340    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7c27d265-67e1-424e-85ff-38a8c30bd547-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-dbd6b58fb-r6mpp\" (UID: \"7c27d265-67e1-424e-85ff-38a8c30bd547\") " pod="openshift-monitoring/telemeter-client-dbd6b58fb-r6mpp"
Oct 30 19:21:52.217274 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.217202    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7c27d265-67e1-424e-85ff-38a8c30bd547-metrics-client-ca\") pod \"telemeter-client-dbd6b58fb-r6mpp\" (UID: \"7c27d265-67e1-424e-85ff-38a8c30bd547\") " pod="openshift-monitoring/telemeter-client-dbd6b58fb-r6mpp"
Oct 30 19:21:52.217606 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.217573    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c27d265-67e1-424e-85ff-38a8c30bd547-serving-certs-ca-bundle\") pod \"telemeter-client-dbd6b58fb-r6mpp\" (UID: \"7c27d265-67e1-424e-85ff-38a8c30bd547\") " pod="openshift-monitoring/telemeter-client-dbd6b58fb-r6mpp"
Oct 30 19:21:52.217694 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.217653    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c27d265-67e1-424e-85ff-38a8c30bd547-telemeter-trusted-ca-bundle\") pod \"telemeter-client-dbd6b58fb-r6mpp\" (UID: \"7c27d265-67e1-424e-85ff-38a8c30bd547\") " pod="openshift-monitoring/telemeter-client-dbd6b58fb-r6mpp"
Oct 30 19:21:52.219107 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.219079    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7c27d265-67e1-424e-85ff-38a8c30bd547-telemeter-client-tls\") pod \"telemeter-client-dbd6b58fb-r6mpp\" (UID: \"7c27d265-67e1-424e-85ff-38a8c30bd547\") " pod="openshift-monitoring/telemeter-client-dbd6b58fb-r6mpp"
Oct 30 19:21:52.219259 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.219239    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7c27d265-67e1-424e-85ff-38a8c30bd547-secret-telemeter-client\") pod \"telemeter-client-dbd6b58fb-r6mpp\" (UID: \"7c27d265-67e1-424e-85ff-38a8c30bd547\") " pod="openshift-monitoring/telemeter-client-dbd6b58fb-r6mpp"
Oct 30 19:21:52.219361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.219345    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7c27d265-67e1-424e-85ff-38a8c30bd547-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-dbd6b58fb-r6mpp\" (UID: \"7c27d265-67e1-424e-85ff-38a8c30bd547\") " pod="openshift-monitoring/telemeter-client-dbd6b58fb-r6mpp"
Oct 30 19:21:52.219466 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.219448    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7c27d265-67e1-424e-85ff-38a8c30bd547-federate-client-tls\") pod \"telemeter-client-dbd6b58fb-r6mpp\" (UID: \"7c27d265-67e1-424e-85ff-38a8c30bd547\") " pod="openshift-monitoring/telemeter-client-dbd6b58fb-r6mpp"
Oct 30 19:21:52.244368 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.244345    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4plf\" (UniqueName: \"kubernetes.io/projected/7c27d265-67e1-424e-85ff-38a8c30bd547-kube-api-access-b4plf\") pod \"telemeter-client-dbd6b58fb-r6mpp\" (UID: \"7c27d265-67e1-424e-85ff-38a8c30bd547\") " pod="openshift-monitoring/telemeter-client-dbd6b58fb-r6mpp"
Oct 30 19:21:52.421967 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.421935    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-dbd6b58fb-r6mpp"
Oct 30 19:21:52.657246 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.657216    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 19:21:52.657713 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.657517    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerName="prometheus" containerID="cri-o://a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63" gracePeriod=600
Oct 30 19:21:52.657713 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.657536    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerName="kube-rbac-proxy-thanos" containerID="cri-o://5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a" gracePeriod=600
Oct 30 19:21:52.657713 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.657558    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerName="thanos-sidecar" containerID="cri-o://79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a" gracePeriod=600
Oct 30 19:21:52.657713 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.657554    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerName="kube-rbac-proxy-web" containerID="cri-o://59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd" gracePeriod=600
Oct 30 19:21:52.657713 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.657588    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerName="kube-rbac-proxy" containerID="cri-o://7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16" gracePeriod=600
Oct 30 19:21:52.657713 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.657597    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerName="config-reloader" containerID="cri-o://67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2" gracePeriod=600
Oct 30 19:21:52.772102 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.771335    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-dbd6b58fb-r6mpp"]
Oct 30 19:21:52.778933 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:52.778898    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c27d265_67e1_424e_85ff_38a8c30bd547.slice/crio-7f7ee30381c8c3442d9a9d6e35864e4c9480b0f91dff6e6746a2b42ec6201105 WatchSource:0}: Error finding container 7f7ee30381c8c3442d9a9d6e35864e4c9480b0f91dff6e6746a2b42ec6201105: Status 404 returned error can't find the container with id 7f7ee30381c8c3442d9a9d6e35864e4c9480b0f91dff6e6746a2b42ec6201105
Oct 30 19:21:52.781041 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.781025    5907 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:21:52.886413 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:21:52.886356    5907 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Canceled desc = context canceled" containerID="a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Oct 30 19:21:52.886582 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:21:52.886449    5907 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Canceled desc = context canceled" containerID="a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Oct 30 19:21:52.886582 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:21:52.886480    5907 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Canceled desc = context canceled" containerID="a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Oct 30 19:21:52.886582 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.886504    5907 prober.go:106] "Probe errored" probeType="Readiness" pod="openshift-monitoring/prometheus-k8s-0" podUID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerName="prometheus" probeResult="unknown" err="rpc error: code = Canceled desc = context canceled"
Oct 30 19:21:52.933755 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:52.933680    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.025139 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.025109    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-thanos-prometheus-http-client-file\") pod \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") "
Oct 30 19:21:53.025303 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.025161    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-configmap-kubelet-serving-ca-bundle\") pod \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") "
Oct 30 19:21:53.025303 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.025181    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-rulefiles-0\") pod \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") "
Oct 30 19:21:53.025303 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.025209    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-prometheus-k8s-tls\") pod \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") "
Oct 30 19:21:53.025460 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.025366    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-web-config\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-web-config\") pod \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") "
Oct 30 19:21:53.025460 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.025438    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-db\") pod \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") "
Oct 30 19:21:53.026063 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.025512    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-rulefiles-1\") pod \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") "
Oct 30 19:21:53.026063 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.025550    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-metrics-client-certs\") pod \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") "
Oct 30 19:21:53.026063 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.025585    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-grpc-tls\") pod \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") "
Oct 30 19:21:53.026063 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.025587    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-configmap-kubelet-serving-ca-bundle" pod "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" (UID: "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:21:53.026063 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.025611    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") "
Oct 30 19:21:53.026063 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.025647    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-kube-rbac-proxy\") pod \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") "
Oct 30 19:21:53.026063 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.025681    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-trusted-ca-bundle\") pod \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") "
Oct 30 19:21:53.026063 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.025714    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") "
Oct 30 19:21:53.026063 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.025740    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-configmap-serving-certs-ca-bundle\") pod \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") "
Oct 30 19:21:53.026063 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.025782    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-tls-assets\" (UniqueName: \"kubernetes.io/projected/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-tls-assets\") pod \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") "
Oct 30 19:21:53.026063 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.025807    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-configmap-metrics-client-ca\") pod \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") "
Oct 30 19:21:53.026063 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.025877    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-rulefiles-2\") pod \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") "
Oct 30 19:21:53.026063 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.025913    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-config-out\" (UniqueName: \"kubernetes.io/empty-dir/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-config-out\") pod \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") "
Oct 30 19:21:53.026063 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.025947    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-kube-api-access-pqn8c\" (UniqueName: \"kubernetes.io/projected/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-kube-api-access-pqn8c\") pod \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") "
Oct 30 19:21:53.026063 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.025979    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-config\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-config\") pod \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\" (UID: \"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963\") "
Oct 30 19:21:53.026816 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.026284    5907 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:53.027429 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.027051    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-rulefiles-1" pod "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" (UID: "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:21:53.029707 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.029669    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-rulefiles-0" pod "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" (UID: "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:21:53.029707 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.029694    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-trusted-ca-bundle" pod "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" (UID: "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:21:53.029707 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.029807    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-db" pod "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" (UID: "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:21:53.030091 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.030063    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-configmap-metrics-client-ca" pod "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" (UID: "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:21:53.030325 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.030281    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-configmap-serving-certs-ca-bundle" pod "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" (UID: "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:21:53.030801 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.030714    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-rulefiles-2" pod "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" (UID: "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:21:53.031084 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.031028    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-thanos-prometheus-http-client-file" pod "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" (UID: "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:21:53.031951 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.031716    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-prometheus-k8s-tls" pod "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" (UID: "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:21:53.031951 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.031907    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-prometheus-k8s-thanos-sidecar-tls" pod "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" (UID: "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:21:53.032109 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.031997    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-kube-rbac-proxy" pod "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" (UID: "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:21:53.032448 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.032411    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-prometheus-k8s-kube-rbac-proxy-web" pod "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" (UID: "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:21:53.032548 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.032527    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-tls-assets" pod "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" (UID: "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:21:53.032603 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.032563    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-config" pod "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" (UID: "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:21:53.032651 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.032619    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-metrics-client-certs" pod "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" (UID: "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:21:53.032830 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.032806    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-grpc-tls" pod "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" (UID: "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:21:53.032830 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.032811    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-config-out" pod "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" (UID: "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:21:53.033031 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.033010    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-kube-api-access-pqn8c" pod "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" (UID: "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963"). InnerVolumeSpecName "kube-api-access-pqn8c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:21:53.041806 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.041784    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-web-config" pod "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" (UID: "ead8f4e0-54d5-4db6-b7fc-6a5ec203a963"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:21:53.127009 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.126971    5907 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-prometheus-k8s-tls\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:53.127009 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.127007    5907 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-web-config\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:53.127240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.127020    5907 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-db\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:53.127240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.127033    5907 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:53.127240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.127046    5907 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-metrics-client-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:53.127240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.127057    5907 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-grpc-tls\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:53.127240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.127071    5907 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:53.127240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.127084    5907 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-kube-rbac-proxy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:53.127240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.127096    5907 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-trusted-ca-bundle\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:53.127240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.127108    5907 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:53.127240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.127122    5907 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:53.127240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.127134    5907 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-tls-assets\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:53.127240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.127148    5907 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-configmap-metrics-client-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:53.127240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.127160    5907 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:53.127240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.127172    5907 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-config-out\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:53.127240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.127184    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqn8c\" (UniqueName: \"kubernetes.io/projected/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-kube-api-access-pqn8c\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:53.127240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.127195    5907 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-config\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:53.127240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.127209    5907 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-thanos-prometheus-http-client-file\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:53.127240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.127223    5907 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:53.482183 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.482054    5907 generic.go:356] "Generic (PLEG): container finished" podID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerID="5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a" exitCode=0
Oct 30 19:21:53.482183 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.482085    5907 generic.go:356] "Generic (PLEG): container finished" podID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerID="7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16" exitCode=0
Oct 30 19:21:53.482183 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.482096    5907 generic.go:356] "Generic (PLEG): container finished" podID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerID="59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd" exitCode=0
Oct 30 19:21:53.482183 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.482104    5907 generic.go:356] "Generic (PLEG): container finished" podID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerID="79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a" exitCode=0
Oct 30 19:21:53.482183 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.482110    5907 generic.go:356] "Generic (PLEG): container finished" podID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerID="67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2" exitCode=0
Oct 30 19:21:53.482183 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.482118    5907 generic.go:356] "Generic (PLEG): container finished" podID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerID="a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63" exitCode=0
Oct 30 19:21:53.482183 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.482143    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.482183 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.482142    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963","Type":"ContainerDied","Data":"5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a"}
Oct 30 19:21:53.482941 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.482224    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963","Type":"ContainerDied","Data":"7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16"}
Oct 30 19:21:53.482941 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.482260    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963","Type":"ContainerDied","Data":"59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd"}
Oct 30 19:21:53.482941 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.482269    5907 scope.go:122] "RemoveContainer" containerID="5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a"
Oct 30 19:21:53.482941 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.482294    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963","Type":"ContainerDied","Data":"79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a"}
Oct 30 19:21:53.482941 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.482337    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963","Type":"ContainerDied","Data":"67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2"}
Oct 30 19:21:53.482941 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.482378    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963","Type":"ContainerDied","Data":"a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63"}
Oct 30 19:21:53.482941 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.482409    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ead8f4e0-54d5-4db6-b7fc-6a5ec203a963","Type":"ContainerDied","Data":"8fe73d8245fc2351fe48fe3f797c40b32b675cb0d0242e738ed1856a41b38627"}
Oct 30 19:21:53.483766 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.483646    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-dbd6b58fb-r6mpp" event={"ID":"7c27d265-67e1-424e-85ff-38a8c30bd547","Type":"ContainerStarted","Data":"7f7ee30381c8c3442d9a9d6e35864e4c9480b0f91dff6e6746a2b42ec6201105"}
Oct 30 19:21:53.491823 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.491793    5907 scope.go:122] "RemoveContainer" containerID="7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16"
Oct 30 19:21:53.500195 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.500173    5907 scope.go:122] "RemoveContainer" containerID="59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd"
Oct 30 19:21:53.508983 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.508956    5907 scope.go:122] "RemoveContainer" containerID="79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a"
Oct 30 19:21:53.517366 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.517344    5907 scope.go:122] "RemoveContainer" containerID="67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2"
Oct 30 19:21:53.525618 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.525585    5907 scope.go:122] "RemoveContainer" containerID="a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63"
Oct 30 19:21:53.535673 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.535642    5907 scope.go:122] "RemoveContainer" containerID="595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed"
Oct 30 19:21:53.541316 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.541294    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 19:21:53.544719 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.544700    5907 scope.go:122] "RemoveContainer" containerID="5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a"
Oct 30 19:21:53.545055 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:21:53.545028    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a\": container with ID starting with 5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a not found: ID does not exist" containerID="5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a"
Oct 30 19:21:53.545156 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.545065    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a"} err="failed to get container status \"5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a\": rpc error: code = NotFound desc = could not find container \"5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a\": container with ID starting with 5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a not found: ID does not exist"
Oct 30 19:21:53.545156 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.545093    5907 scope.go:122] "RemoveContainer" containerID="7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16"
Oct 30 19:21:53.545362 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:21:53.545334    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16\": container with ID starting with 7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16 not found: ID does not exist" containerID="7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16"
Oct 30 19:21:53.545461 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.545372    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16"} err="failed to get container status \"7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16\": rpc error: code = NotFound desc = could not find container \"7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16\": container with ID starting with 7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16 not found: ID does not exist"
Oct 30 19:21:53.545461 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.545397    5907 scope.go:122] "RemoveContainer" containerID="59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd"
Oct 30 19:21:53.545845 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:21:53.545826    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd\": container with ID starting with 59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd not found: ID does not exist" containerID="59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd"
Oct 30 19:21:53.545951 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.545850    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd"} err="failed to get container status \"59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd\": rpc error: code = NotFound desc = could not find container \"59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd\": container with ID starting with 59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd not found: ID does not exist"
Oct 30 19:21:53.545951 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.545883    5907 scope.go:122] "RemoveContainer" containerID="79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a"
Oct 30 19:21:53.546234 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:21:53.546209    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a\": container with ID starting with 79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a not found: ID does not exist" containerID="79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a"
Oct 30 19:21:53.546328 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.546242    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a"} err="failed to get container status \"79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a\": rpc error: code = NotFound desc = could not find container \"79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a\": container with ID starting with 79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a not found: ID does not exist"
Oct 30 19:21:53.546328 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.546262    5907 scope.go:122] "RemoveContainer" containerID="67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2"
Oct 30 19:21:53.546552 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:21:53.546526    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2\": container with ID starting with 67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2 not found: ID does not exist" containerID="67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2"
Oct 30 19:21:53.546642 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.546560    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2"} err="failed to get container status \"67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2\": rpc error: code = NotFound desc = could not find container \"67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2\": container with ID starting with 67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2 not found: ID does not exist"
Oct 30 19:21:53.546642 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.546577    5907 scope.go:122] "RemoveContainer" containerID="a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63"
Oct 30 19:21:53.546812 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:21:53.546777    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63\": container with ID starting with a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63 not found: ID does not exist" containerID="a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63"
Oct 30 19:21:53.546915 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.546807    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63"} err="failed to get container status \"a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63\": rpc error: code = NotFound desc = could not find container \"a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63\": container with ID starting with a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63 not found: ID does not exist"
Oct 30 19:21:53.546915 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.546828    5907 scope.go:122] "RemoveContainer" containerID="595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed"
Oct 30 19:21:53.547146 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:21:53.547127    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed\": container with ID starting with 595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed not found: ID does not exist" containerID="595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed"
Oct 30 19:21:53.547222 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.547153    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed"} err="failed to get container status \"595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed\": rpc error: code = NotFound desc = could not find container \"595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed\": container with ID starting with 595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed not found: ID does not exist"
Oct 30 19:21:53.547222 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.547173    5907 scope.go:122] "RemoveContainer" containerID="5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a"
Oct 30 19:21:53.547442 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.547423    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a"} err="failed to get container status \"5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a\": rpc error: code = NotFound desc = could not find container \"5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a\": container with ID starting with 5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a not found: ID does not exist"
Oct 30 19:21:53.547496 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.547443    5907 scope.go:122] "RemoveContainer" containerID="7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16"
Oct 30 19:21:53.547671 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.547644    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16"} err="failed to get container status \"7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16\": rpc error: code = NotFound desc = could not find container \"7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16\": container with ID starting with 7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16 not found: ID does not exist"
Oct 30 19:21:53.547671 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.547671    5907 scope.go:122] "RemoveContainer" containerID="59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd"
Oct 30 19:21:53.547969 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.547939    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd"} err="failed to get container status \"59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd\": rpc error: code = NotFound desc = could not find container \"59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd\": container with ID starting with 59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd not found: ID does not exist"
Oct 30 19:21:53.547969 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.547960    5907 scope.go:122] "RemoveContainer" containerID="79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a"
Oct 30 19:21:53.548227 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.548195    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a"} err="failed to get container status \"79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a\": rpc error: code = NotFound desc = could not find container \"79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a\": container with ID starting with 79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a not found: ID does not exist"
Oct 30 19:21:53.548227 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.548222    5907 scope.go:122] "RemoveContainer" containerID="67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2"
Oct 30 19:21:53.548512 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.548491    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2"} err="failed to get container status \"67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2\": rpc error: code = NotFound desc = could not find container \"67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2\": container with ID starting with 67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2 not found: ID does not exist"
Oct 30 19:21:53.548584 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.548512    5907 scope.go:122] "RemoveContainer" containerID="a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63"
Oct 30 19:21:53.548767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.548745    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63"} err="failed to get container status \"a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63\": rpc error: code = NotFound desc = could not find container \"a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63\": container with ID starting with a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63 not found: ID does not exist"
Oct 30 19:21:53.548816 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.548770    5907 scope.go:122] "RemoveContainer" containerID="595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed"
Oct 30 19:21:53.549066 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.549044    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed"} err="failed to get container status \"595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed\": rpc error: code = NotFound desc = could not find container \"595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed\": container with ID starting with 595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed not found: ID does not exist"
Oct 30 19:21:53.549066 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.549066    5907 scope.go:122] "RemoveContainer" containerID="5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a"
Oct 30 19:21:53.549311 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.549291    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a"} err="failed to get container status \"5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a\": rpc error: code = NotFound desc = could not find container \"5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a\": container with ID starting with 5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a not found: ID does not exist"
Oct 30 19:21:53.549378 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.549312    5907 scope.go:122] "RemoveContainer" containerID="7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16"
Oct 30 19:21:53.549584 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.549555    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16"} err="failed to get container status \"7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16\": rpc error: code = NotFound desc = could not find container \"7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16\": container with ID starting with 7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16 not found: ID does not exist"
Oct 30 19:21:53.549584 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.549581    5907 scope.go:122] "RemoveContainer" containerID="59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd"
Oct 30 19:21:53.549906 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.549845    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd"} err="failed to get container status \"59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd\": rpc error: code = NotFound desc = could not find container \"59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd\": container with ID starting with 59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd not found: ID does not exist"
Oct 30 19:21:53.549906 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.549886    5907 scope.go:122] "RemoveContainer" containerID="79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a"
Oct 30 19:21:53.550177 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.550157    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a"} err="failed to get container status \"79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a\": rpc error: code = NotFound desc = could not find container \"79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a\": container with ID starting with 79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a not found: ID does not exist"
Oct 30 19:21:53.550287 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.550177    5907 scope.go:122] "RemoveContainer" containerID="67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2"
Oct 30 19:21:53.550473 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.550444    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2"} err="failed to get container status \"67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2\": rpc error: code = NotFound desc = could not find container \"67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2\": container with ID starting with 67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2 not found: ID does not exist"
Oct 30 19:21:53.550575 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.550473    5907 scope.go:122] "RemoveContainer" containerID="a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63"
Oct 30 19:21:53.550845 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.550824    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63"} err="failed to get container status \"a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63\": rpc error: code = NotFound desc = could not find container \"a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63\": container with ID starting with a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63 not found: ID does not exist"
Oct 30 19:21:53.550954 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.550846    5907 scope.go:122] "RemoveContainer" containerID="595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed"
Oct 30 19:21:53.551090 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.551073    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed"} err="failed to get container status \"595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed\": rpc error: code = NotFound desc = could not find container \"595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed\": container with ID starting with 595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed not found: ID does not exist"
Oct 30 19:21:53.551157 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.551092    5907 scope.go:122] "RemoveContainer" containerID="5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a"
Oct 30 19:21:53.551332 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.551312    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a"} err="failed to get container status \"5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a\": rpc error: code = NotFound desc = could not find container \"5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a\": container with ID starting with 5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a not found: ID does not exist"
Oct 30 19:21:53.551396 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.551333    5907 scope.go:122] "RemoveContainer" containerID="7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16"
Oct 30 19:21:53.551621 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.551583    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16"} err="failed to get container status \"7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16\": rpc error: code = NotFound desc = could not find container \"7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16\": container with ID starting with 7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16 not found: ID does not exist"
Oct 30 19:21:53.551764 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.551620    5907 scope.go:122] "RemoveContainer" containerID="59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd"
Oct 30 19:21:53.551907 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.551888    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd"} err="failed to get container status \"59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd\": rpc error: code = NotFound desc = could not find container \"59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd\": container with ID starting with 59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd not found: ID does not exist"
Oct 30 19:21:53.551996 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.551909    5907 scope.go:122] "RemoveContainer" containerID="79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a"
Oct 30 19:21:53.552395 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.552376    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a"} err="failed to get container status \"79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a\": rpc error: code = NotFound desc = could not find container \"79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a\": container with ID starting with 79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a not found: ID does not exist"
Oct 30 19:21:53.552481 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.552396    5907 scope.go:122] "RemoveContainer" containerID="67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2"
Oct 30 19:21:53.552702 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.552678    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2"} err="failed to get container status \"67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2\": rpc error: code = NotFound desc = could not find container \"67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2\": container with ID starting with 67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2 not found: ID does not exist"
Oct 30 19:21:53.552702 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.552703    5907 scope.go:122] "RemoveContainer" containerID="a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63"
Oct 30 19:21:53.552977 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.552958    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63"} err="failed to get container status \"a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63\": rpc error: code = NotFound desc = could not find container \"a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63\": container with ID starting with a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63 not found: ID does not exist"
Oct 30 19:21:53.552977 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.552978    5907 scope.go:122] "RemoveContainer" containerID="595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed"
Oct 30 19:21:53.553240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.553215    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed"} err="failed to get container status \"595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed\": rpc error: code = NotFound desc = could not find container \"595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed\": container with ID starting with 595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed not found: ID does not exist"
Oct 30 19:21:53.553322 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.553240    5907 scope.go:122] "RemoveContainer" containerID="5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a"
Oct 30 19:21:53.553979 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.553838    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a"} err="failed to get container status \"5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a\": rpc error: code = NotFound desc = could not find container \"5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a\": container with ID starting with 5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a not found: ID does not exist"
Oct 30 19:21:53.553979 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.553891    5907 scope.go:122] "RemoveContainer" containerID="7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16"
Oct 30 19:21:53.554366 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.554338    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16"} err="failed to get container status \"7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16\": rpc error: code = NotFound desc = could not find container \"7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16\": container with ID starting with 7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16 not found: ID does not exist"
Oct 30 19:21:53.554464 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.554367    5907 scope.go:122] "RemoveContainer" containerID="59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd"
Oct 30 19:21:53.554683 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.554663    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd"} err="failed to get container status \"59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd\": rpc error: code = NotFound desc = could not find container \"59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd\": container with ID starting with 59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd not found: ID does not exist"
Oct 30 19:21:53.554748 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.554683    5907 scope.go:122] "RemoveContainer" containerID="79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a"
Oct 30 19:21:53.554962 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.554943    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a"} err="failed to get container status \"79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a\": rpc error: code = NotFound desc = could not find container \"79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a\": container with ID starting with 79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a not found: ID does not exist"
Oct 30 19:21:53.554962 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.554962    5907 scope.go:122] "RemoveContainer" containerID="67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2"
Oct 30 19:21:53.555240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.555216    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2"} err="failed to get container status \"67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2\": rpc error: code = NotFound desc = could not find container \"67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2\": container with ID starting with 67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2 not found: ID does not exist"
Oct 30 19:21:53.555322 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.555241    5907 scope.go:122] "RemoveContainer" containerID="a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63"
Oct 30 19:21:53.555638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.555609    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63"} err="failed to get container status \"a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63\": rpc error: code = NotFound desc = could not find container \"a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63\": container with ID starting with a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63 not found: ID does not exist"
Oct 30 19:21:53.555638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.555638    5907 scope.go:122] "RemoveContainer" containerID="595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed"
Oct 30 19:21:53.556770 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.556747    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 19:21:53.556942 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.556917    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed"} err="failed to get container status \"595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed\": rpc error: code = NotFound desc = could not find container \"595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed\": container with ID starting with 595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed not found: ID does not exist"
Oct 30 19:21:53.557028 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.556948    5907 scope.go:122] "RemoveContainer" containerID="5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a"
Oct 30 19:21:53.557254 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.557235    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a"} err="failed to get container status \"5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a\": rpc error: code = NotFound desc = could not find container \"5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a\": container with ID starting with 5588c8dd36bb2321b9fedc808c31351ca391b9b9c347cc0252a40b16acffe83a not found: ID does not exist"
Oct 30 19:21:53.557320 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.557255    5907 scope.go:122] "RemoveContainer" containerID="7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16"
Oct 30 19:21:53.557564 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.557537    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16"} err="failed to get container status \"7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16\": rpc error: code = NotFound desc = could not find container \"7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16\": container with ID starting with 7af059a8c4041f479afafcd0aad085cf0237fec4d59de07e43cf84cef707ac16 not found: ID does not exist"
Oct 30 19:21:53.557656 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.557566    5907 scope.go:122] "RemoveContainer" containerID="59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd"
Oct 30 19:21:53.557910 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.557887    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd"} err="failed to get container status \"59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd\": rpc error: code = NotFound desc = could not find container \"59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd\": container with ID starting with 59264b568491e6b38b06ef4c5c2f76f3b0b3ba01d50870efac10664395f27edd not found: ID does not exist"
Oct 30 19:21:53.558013 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.557911    5907 scope.go:122] "RemoveContainer" containerID="79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a"
Oct 30 19:21:53.558205 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.558186    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a"} err="failed to get container status \"79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a\": rpc error: code = NotFound desc = could not find container \"79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a\": container with ID starting with 79efe83a65f4ccaaa24232e0d4399c38a3cad84abfd9a8f6565e77d366e2498a not found: ID does not exist"
Oct 30 19:21:53.558280 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.558205    5907 scope.go:122] "RemoveContainer" containerID="67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2"
Oct 30 19:21:53.558530 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.558508    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2"} err="failed to get container status \"67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2\": rpc error: code = NotFound desc = could not find container \"67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2\": container with ID starting with 67225f84b4cf6f43240fe84b860f11277f32ad89540cfe4d3ba4aa0c3f97c9a2 not found: ID does not exist"
Oct 30 19:21:53.558602 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.558531    5907 scope.go:122] "RemoveContainer" containerID="a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63"
Oct 30 19:21:53.558827 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.558796    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63"} err="failed to get container status \"a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63\": rpc error: code = NotFound desc = could not find container \"a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63\": container with ID starting with a29763a57f56163bdf4f8a096af617b287c125b18486601088742dcb1f5f3a63 not found: ID does not exist"
Oct 30 19:21:53.558827 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.558822    5907 scope.go:122] "RemoveContainer" containerID="595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed"
Oct 30 19:21:53.559115 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.559096    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed"} err="failed to get container status \"595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed\": rpc error: code = NotFound desc = could not find container \"595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed\": container with ID starting with 595736474d2b50dba3f6460faa6457bcab01f43b3a40496363ecc86ebe6ad9ed not found: ID does not exist"
Oct 30 19:21:53.572277 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.572244    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 19:21:53.572698 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.572679    5907 cpu_manager.go:414] "containerMap: removing container" podUID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerName="kube-rbac-proxy"
Oct 30 19:21:53.572698 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.572697    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerName="kube-rbac-proxy"
Oct 30 19:21:53.572822 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.572713    5907 cpu_manager.go:414] "containerMap: removing container" podUID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerName="kube-rbac-proxy-web"
Oct 30 19:21:53.572822 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.572718    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerName="kube-rbac-proxy-web"
Oct 30 19:21:53.572822 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.572725    5907 cpu_manager.go:414] "containerMap: removing container" podUID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerName="thanos-sidecar"
Oct 30 19:21:53.572822 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.572729    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerName="thanos-sidecar"
Oct 30 19:21:53.572822 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.572735    5907 cpu_manager.go:414] "containerMap: removing container" podUID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerName="init-config-reloader"
Oct 30 19:21:53.572822 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.572739    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerName="init-config-reloader"
Oct 30 19:21:53.572822 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.572746    5907 cpu_manager.go:414] "containerMap: removing container" podUID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerName="config-reloader"
Oct 30 19:21:53.572822 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.572750    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerName="config-reloader"
Oct 30 19:21:53.572822 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.572761    5907 cpu_manager.go:414] "containerMap: removing container" podUID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerName="prometheus"
Oct 30 19:21:53.572822 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.572766    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerName="prometheus"
Oct 30 19:21:53.572822 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.572776    5907 cpu_manager.go:414] "containerMap: removing container" podUID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerName="kube-rbac-proxy-thanos"
Oct 30 19:21:53.572822 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.572781    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerName="kube-rbac-proxy-thanos"
Oct 30 19:21:53.573181 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.572835    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerName="kube-rbac-proxy-web"
Oct 30 19:21:53.573181 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.572845    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerName="thanos-sidecar"
Oct 30 19:21:53.573181 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.572853    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerName="init-config-reloader"
Oct 30 19:21:53.573181 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.572885    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerName="config-reloader"
Oct 30 19:21:53.573181 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.572893    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerName="prometheus"
Oct 30 19:21:53.573181 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.572899    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerName="kube-rbac-proxy-thanos"
Oct 30 19:21:53.573181 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.572908    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" containerName="kube-rbac-proxy"
Oct 30 19:21:53.592400 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.592376    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.604142 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.604107    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 19:21:53.611106 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.611080    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 19:21:53.611252 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.611237    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 19:21:53.616586 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.616554    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 19:21:53.623832 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.623785    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 19:21:53.623832 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.623789    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 19:21:53.624023 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.623941    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 19:21:53.624023 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.623961    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 19:21:53.624023 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.623977    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 19:21:53.624023 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.623975    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 19:21:53.624023 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.623960    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 19:21:53.624023 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.623977    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 19:21:53.624734 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.624719    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7380srhp40v4j\""
Oct 30 19:21:53.624996 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.624973    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 19:21:53.625215 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.625164    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-cqcj8\""
Oct 30 19:21:53.625215 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.625171    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 19:21:53.625678 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.625661    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 19:21:53.631934 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.631911    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/aa815a19-9652-4492-9d04-5aa25725131c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.632030 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.631952    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/aa815a19-9652-4492-9d04-5aa25725131c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.632030 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.631976    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/aa815a19-9652-4492-9d04-5aa25725131c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.632153 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.632081    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa815a19-9652-4492-9d04-5aa25725131c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.632153 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.632108    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/aa815a19-9652-4492-9d04-5aa25725131c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.632153 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.632135    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aa815a19-9652-4492-9d04-5aa25725131c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.632287 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.632257    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa815a19-9652-4492-9d04-5aa25725131c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.632341 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.632292    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa815a19-9652-4492-9d04-5aa25725131c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.632341 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.632310    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa815a19-9652-4492-9d04-5aa25725131c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.632435 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.632342    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/aa815a19-9652-4492-9d04-5aa25725131c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.632435 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.632422    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/aa815a19-9652-4492-9d04-5aa25725131c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.632513 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.632448    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/aa815a19-9652-4492-9d04-5aa25725131c-config\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.632513 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.632464    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aa815a19-9652-4492-9d04-5aa25725131c-config-out\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.632584 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.632516    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa815a19-9652-4492-9d04-5aa25725131c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.632584 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.632544    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqkst\" (UniqueName: \"kubernetes.io/projected/aa815a19-9652-4492-9d04-5aa25725131c-kube-api-access-tqkst\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.632685 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.632667    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/aa815a19-9652-4492-9d04-5aa25725131c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.632814 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.632703    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/aa815a19-9652-4492-9d04-5aa25725131c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.632814 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.632729    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aa815a19-9652-4492-9d04-5aa25725131c-web-config\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.632814 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.632801    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa815a19-9652-4492-9d04-5aa25725131c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.633136 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.632833    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/aa815a19-9652-4492-9d04-5aa25725131c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.733813 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.733736    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/aa815a19-9652-4492-9d04-5aa25725131c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.733813 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.733781    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/aa815a19-9652-4492-9d04-5aa25725131c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.733813 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.733808    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aa815a19-9652-4492-9d04-5aa25725131c-web-config\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.734364 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.733830    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa815a19-9652-4492-9d04-5aa25725131c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.734364 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.733855    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/aa815a19-9652-4492-9d04-5aa25725131c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.734364 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.733896    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/aa815a19-9652-4492-9d04-5aa25725131c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.734364 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.733919    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/aa815a19-9652-4492-9d04-5aa25725131c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.734364 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.733934    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/aa815a19-9652-4492-9d04-5aa25725131c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.734364 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.733950    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa815a19-9652-4492-9d04-5aa25725131c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.734364 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.733989    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/aa815a19-9652-4492-9d04-5aa25725131c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.734364 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.734016    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aa815a19-9652-4492-9d04-5aa25725131c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.734364 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.734051    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa815a19-9652-4492-9d04-5aa25725131c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.734364 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.734079    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa815a19-9652-4492-9d04-5aa25725131c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.734364 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.734103    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa815a19-9652-4492-9d04-5aa25725131c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.734364 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.734137    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/aa815a19-9652-4492-9d04-5aa25725131c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.734364 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.734163    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/aa815a19-9652-4492-9d04-5aa25725131c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.734364 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.734202    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/aa815a19-9652-4492-9d04-5aa25725131c-config\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.734364 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.734219    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aa815a19-9652-4492-9d04-5aa25725131c-config-out\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.734364 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.734236    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa815a19-9652-4492-9d04-5aa25725131c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.734364 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.734256    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqkst\" (UniqueName: \"kubernetes.io/projected/aa815a19-9652-4492-9d04-5aa25725131c-kube-api-access-tqkst\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.735413 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.735388    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa815a19-9652-4492-9d04-5aa25725131c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.736222 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.735906    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa815a19-9652-4492-9d04-5aa25725131c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.737124 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.736722    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/aa815a19-9652-4492-9d04-5aa25725131c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.738665 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.737903    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/aa815a19-9652-4492-9d04-5aa25725131c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.738665 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.737910    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa815a19-9652-4492-9d04-5aa25725131c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.738665 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.738201    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/aa815a19-9652-4492-9d04-5aa25725131c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.738665 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.738468    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/aa815a19-9652-4492-9d04-5aa25725131c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.739707 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.739211    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa815a19-9652-4492-9d04-5aa25725131c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.739707 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.739304    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/aa815a19-9652-4492-9d04-5aa25725131c-config\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.739707 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.739659    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/aa815a19-9652-4492-9d04-5aa25725131c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.739976 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.739953    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aa815a19-9652-4492-9d04-5aa25725131c-web-config\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.740344 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.740317    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aa815a19-9652-4492-9d04-5aa25725131c-config-out\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.740424 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.740360    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/aa815a19-9652-4492-9d04-5aa25725131c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.741027 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.741003    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa815a19-9652-4492-9d04-5aa25725131c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.741797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.741777    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aa815a19-9652-4492-9d04-5aa25725131c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.742074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.742032    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/aa815a19-9652-4492-9d04-5aa25725131c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.742163 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.742148    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/aa815a19-9652-4492-9d04-5aa25725131c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.742570 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.742545    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/aa815a19-9652-4492-9d04-5aa25725131c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.743263 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.743240    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa815a19-9652-4492-9d04-5aa25725131c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.756087 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.756058    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqkst\" (UniqueName: \"kubernetes.io/projected/aa815a19-9652-4492-9d04-5aa25725131c-kube-api-access-tqkst\") pod \"prometheus-k8s-0\" (UID: \"aa815a19-9652-4492-9d04-5aa25725131c\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:53.992304 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:53.992221    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:21:54.287195 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:54.287119    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ead8f4e0-54d5-4db6-b7fc-6a5ec203a963" path="/var/lib/kubelet/pods/ead8f4e0-54d5-4db6-b7fc-6a5ec203a963/volumes"
Oct 30 19:21:54.315498 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:54.315458    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-5d79db5f98-fmng5" podUID="44b9fc01-6837-46cd-bf01-40cb8fcb90f5" containerName="console" containerID="cri-o://213d8582e8b2772f3ea9d89844f4f2694c46a2701148a17cd72a3317666313c7" gracePeriod=15
Oct 30 19:21:54.356760 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:54.356736    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 19:21:54.427920 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:54.427885    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa815a19_9652_4492_9d04_5aa25725131c.slice/crio-162cbb1dae79c06779fd7e7617f27439dccba7d230cfe4f5cecf6126ea75d6e6 WatchSource:0}: Error finding container 162cbb1dae79c06779fd7e7617f27439dccba7d230cfe4f5cecf6126ea75d6e6: Status 404 returned error can't find the container with id 162cbb1dae79c06779fd7e7617f27439dccba7d230cfe4f5cecf6126ea75d6e6
Oct 30 19:21:54.489632 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:54.489602    5907 generic.go:356] "Generic (PLEG): container finished" podID="44b9fc01-6837-46cd-bf01-40cb8fcb90f5" containerID="213d8582e8b2772f3ea9d89844f4f2694c46a2701148a17cd72a3317666313c7" exitCode=0
Oct 30 19:21:54.489823 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:54.489688    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d79db5f98-fmng5" event={"ID":"44b9fc01-6837-46cd-bf01-40cb8fcb90f5","Type":"ContainerDied","Data":"213d8582e8b2772f3ea9d89844f4f2694c46a2701148a17cd72a3317666313c7"}
Oct 30 19:21:54.490974 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:54.490948    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa815a19-9652-4492-9d04-5aa25725131c","Type":"ContainerStarted","Data":"162cbb1dae79c06779fd7e7617f27439dccba7d230cfe4f5cecf6126ea75d6e6"}
Oct 30 19:21:54.641441 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:54.641417    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:54.744483 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:54.744447    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-console-serving-cert\") pod \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") "
Oct 30 19:21:54.744970 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:54.744496    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-oauth-serving-cert\") pod \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") "
Oct 30 19:21:54.744970 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:54.744620    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-service-ca\" (UniqueName: \"kubernetes.io/configmap/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-service-ca\") pod \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") "
Oct 30 19:21:54.744970 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:54.744665    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-kube-api-access-6klk6\" (UniqueName: \"kubernetes.io/projected/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-kube-api-access-6klk6\") pod \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") "
Oct 30 19:21:54.744970 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:54.744690    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-console-oauth-config\") pod \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") "
Oct 30 19:21:54.744970 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:54.744719    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-console-config\" (UniqueName: \"kubernetes.io/configmap/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-console-config\") pod \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") "
Oct 30 19:21:54.744970 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:54.744742    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-tmp\" (UniqueName: \"kubernetes.io/empty-dir/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-tmp\") pod \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\" (UID: \"44b9fc01-6837-46cd-bf01-40cb8fcb90f5\") "
Oct 30 19:21:54.744970 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:54.744910    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-oauth-serving-cert" pod "44b9fc01-6837-46cd-bf01-40cb8fcb90f5" (UID: "44b9fc01-6837-46cd-bf01-40cb8fcb90f5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:21:54.745288 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:54.745113    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-tmp" pod "44b9fc01-6837-46cd-bf01-40cb8fcb90f5" (UID: "44b9fc01-6837-46cd-bf01-40cb8fcb90f5"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:21:54.745288 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:54.745141    5907 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-oauth-serving-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:54.745288 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:54.745153    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-service-ca" pod "44b9fc01-6837-46cd-bf01-40cb8fcb90f5" (UID: "44b9fc01-6837-46cd-bf01-40cb8fcb90f5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:21:54.745288 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:54.745163    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-console-config" pod "44b9fc01-6837-46cd-bf01-40cb8fcb90f5" (UID: "44b9fc01-6837-46cd-bf01-40cb8fcb90f5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:21:54.747198 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:54.747175    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-console-oauth-config" pod "44b9fc01-6837-46cd-bf01-40cb8fcb90f5" (UID: "44b9fc01-6837-46cd-bf01-40cb8fcb90f5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:21:54.747465 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:54.747447    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-console-serving-cert" pod "44b9fc01-6837-46cd-bf01-40cb8fcb90f5" (UID: "44b9fc01-6837-46cd-bf01-40cb8fcb90f5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:21:54.747510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:54.747457    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-kube-api-access-6klk6" pod "44b9fc01-6837-46cd-bf01-40cb8fcb90f5" (UID: "44b9fc01-6837-46cd-bf01-40cb8fcb90f5"). InnerVolumeSpecName "kube-api-access-6klk6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:21:54.845966 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:54.845934    5907 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-service-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:54.846063 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:54.845971    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6klk6\" (UniqueName: \"kubernetes.io/projected/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-kube-api-access-6klk6\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:54.846063 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:54.845987    5907 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-console-oauth-config\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:54.846063 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:54.846001    5907 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-console-config\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:54.846063 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:54.846017    5907 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-tmp\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:54.846063 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:54.846031    5907 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/44b9fc01-6837-46cd-bf01-40cb8fcb90f5-console-serving-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:21:55.496933 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.496827    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d79db5f98-fmng5"
Oct 30 19:21:55.496933 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.496830    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d79db5f98-fmng5" event={"ID":"44b9fc01-6837-46cd-bf01-40cb8fcb90f5","Type":"ContainerDied","Data":"9f818a927ce3a7d5f3fe85edc4285eb3f0932cc7b6bfc950eec0956ed2e70d7f"}
Oct 30 19:21:55.496933 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.496922    5907 scope.go:122] "RemoveContainer" containerID="213d8582e8b2772f3ea9d89844f4f2694c46a2701148a17cd72a3317666313c7"
Oct 30 19:21:55.498698 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.498667    5907 generic.go:356] "Generic (PLEG): container finished" podID="aa815a19-9652-4492-9d04-5aa25725131c" containerID="1604f851cab54446303f4416c0cc512937c3635e6919ea7326d5d5a1658bbfd8" exitCode=0
Oct 30 19:21:55.498833 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.498756    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa815a19-9652-4492-9d04-5aa25725131c","Type":"ContainerDied","Data":"1604f851cab54446303f4416c0cc512937c3635e6919ea7326d5d5a1658bbfd8"}
Oct 30 19:21:55.501343 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.501310    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-dbd6b58fb-r6mpp" event={"ID":"7c27d265-67e1-424e-85ff-38a8c30bd547","Type":"ContainerStarted","Data":"4b2b7843277453ba23e1cd0273863f5be807966168afb58b72d76b3302a23245"}
Oct 30 19:21:55.501460 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.501366    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-dbd6b58fb-r6mpp" event={"ID":"7c27d265-67e1-424e-85ff-38a8c30bd547","Type":"ContainerStarted","Data":"22ea345df7177a27f46aea3807aebfb207a5ed613c591214f482f77777ac94cf"}
Oct 30 19:21:55.501460 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.501405    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-dbd6b58fb-r6mpp" event={"ID":"7c27d265-67e1-424e-85ff-38a8c30bd547","Type":"ContainerStarted","Data":"ee6f90a253584f1e39266c557cc7bda210f84fad67d1df9f00194365f3b2538d"}
Oct 30 19:21:55.558297 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.558249    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-dbd6b58fb-r6mpp" podStartSLOduration=1.5861251859999999 podStartE2EDuration="3.558236047s" totalImagesPullingTime="1.972110861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:55.556921827 +0000 UTC m=+27.399853767" watchObservedRunningTime="2026-10-30 19:21:55.558236047 +0000 UTC m=+27.401167983"
Oct 30 19:21:55.646594 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.646565    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d79db5f98-fmng5"]
Oct 30 19:21:55.648851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.648826    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5d79db5f98-fmng5"]
Oct 30 19:21:55.711525 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.711484    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-b966cd7f8-6pc6x"]
Oct 30 19:21:55.712029 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.712010    5907 cpu_manager.go:414] "containerMap: removing container" podUID="44b9fc01-6837-46cd-bf01-40cb8fcb90f5" containerName="console"
Oct 30 19:21:55.712123 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.712032    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="44b9fc01-6837-46cd-bf01-40cb8fcb90f5" containerName="console"
Oct 30 19:21:55.712123 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.712116    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="44b9fc01-6837-46cd-bf01-40cb8fcb90f5" containerName="console"
Oct 30 19:21:55.737151 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.737124    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b966cd7f8-6pc6x"
Oct 30 19:21:55.739080 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.739050    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b966cd7f8-6pc6x"]
Oct 30 19:21:55.755511 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.755186    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a322374a-1a5a-45ae-95b3-454613387e51-console-config\") pod \"console-b966cd7f8-6pc6x\" (UID: \"a322374a-1a5a-45ae-95b3-454613387e51\") " pod="openshift-console/console-b966cd7f8-6pc6x"
Oct 30 19:21:55.760177 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.759824    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a322374a-1a5a-45ae-95b3-454613387e51-service-ca\") pod \"console-b966cd7f8-6pc6x\" (UID: \"a322374a-1a5a-45ae-95b3-454613387e51\") " pod="openshift-console/console-b966cd7f8-6pc6x"
Oct 30 19:21:55.760415 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.760382    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pxxl\" (UniqueName: \"kubernetes.io/projected/a322374a-1a5a-45ae-95b3-454613387e51-kube-api-access-4pxxl\") pod \"console-b966cd7f8-6pc6x\" (UID: \"a322374a-1a5a-45ae-95b3-454613387e51\") " pod="openshift-console/console-b966cd7f8-6pc6x"
Oct 30 19:21:55.760646 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.760627    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a322374a-1a5a-45ae-95b3-454613387e51-tmp\") pod \"console-b966cd7f8-6pc6x\" (UID: \"a322374a-1a5a-45ae-95b3-454613387e51\") " pod="openshift-console/console-b966cd7f8-6pc6x"
Oct 30 19:21:55.761187 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.761160    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a322374a-1a5a-45ae-95b3-454613387e51-console-oauth-config\") pod \"console-b966cd7f8-6pc6x\" (UID: \"a322374a-1a5a-45ae-95b3-454613387e51\") " pod="openshift-console/console-b966cd7f8-6pc6x"
Oct 30 19:21:55.761289 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.761212    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a322374a-1a5a-45ae-95b3-454613387e51-oauth-serving-cert\") pod \"console-b966cd7f8-6pc6x\" (UID: \"a322374a-1a5a-45ae-95b3-454613387e51\") " pod="openshift-console/console-b966cd7f8-6pc6x"
Oct 30 19:21:55.761289 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.761255    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a322374a-1a5a-45ae-95b3-454613387e51-console-serving-cert\") pod \"console-b966cd7f8-6pc6x\" (UID: \"a322374a-1a5a-45ae-95b3-454613387e51\") " pod="openshift-console/console-b966cd7f8-6pc6x"
Oct 30 19:21:55.761400 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.761386    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a322374a-1a5a-45ae-95b3-454613387e51-trusted-ca-bundle\") pod \"console-b966cd7f8-6pc6x\" (UID: \"a322374a-1a5a-45ae-95b3-454613387e51\") " pod="openshift-console/console-b966cd7f8-6pc6x"
Oct 30 19:21:55.862388 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.862353    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a322374a-1a5a-45ae-95b3-454613387e51-service-ca\") pod \"console-b966cd7f8-6pc6x\" (UID: \"a322374a-1a5a-45ae-95b3-454613387e51\") " pod="openshift-console/console-b966cd7f8-6pc6x"
Oct 30 19:21:55.862513 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.862406    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pxxl\" (UniqueName: \"kubernetes.io/projected/a322374a-1a5a-45ae-95b3-454613387e51-kube-api-access-4pxxl\") pod \"console-b966cd7f8-6pc6x\" (UID: \"a322374a-1a5a-45ae-95b3-454613387e51\") " pod="openshift-console/console-b966cd7f8-6pc6x"
Oct 30 19:21:55.862513 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.862447    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a322374a-1a5a-45ae-95b3-454613387e51-tmp\") pod \"console-b966cd7f8-6pc6x\" (UID: \"a322374a-1a5a-45ae-95b3-454613387e51\") " pod="openshift-console/console-b966cd7f8-6pc6x"
Oct 30 19:21:55.862513 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.862498    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a322374a-1a5a-45ae-95b3-454613387e51-console-oauth-config\") pod \"console-b966cd7f8-6pc6x\" (UID: \"a322374a-1a5a-45ae-95b3-454613387e51\") " pod="openshift-console/console-b966cd7f8-6pc6x"
Oct 30 19:21:55.862616 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.862534    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a322374a-1a5a-45ae-95b3-454613387e51-oauth-serving-cert\") pod \"console-b966cd7f8-6pc6x\" (UID: \"a322374a-1a5a-45ae-95b3-454613387e51\") " pod="openshift-console/console-b966cd7f8-6pc6x"
Oct 30 19:21:55.862616 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.862572    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a322374a-1a5a-45ae-95b3-454613387e51-console-serving-cert\") pod \"console-b966cd7f8-6pc6x\" (UID: \"a322374a-1a5a-45ae-95b3-454613387e51\") " pod="openshift-console/console-b966cd7f8-6pc6x"
Oct 30 19:21:55.862616 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.862610    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a322374a-1a5a-45ae-95b3-454613387e51-trusted-ca-bundle\") pod \"console-b966cd7f8-6pc6x\" (UID: \"a322374a-1a5a-45ae-95b3-454613387e51\") " pod="openshift-console/console-b966cd7f8-6pc6x"
Oct 30 19:21:55.862704 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.862669    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a322374a-1a5a-45ae-95b3-454613387e51-console-config\") pod \"console-b966cd7f8-6pc6x\" (UID: \"a322374a-1a5a-45ae-95b3-454613387e51\") " pod="openshift-console/console-b966cd7f8-6pc6x"
Oct 30 19:21:55.863319 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.863295    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a322374a-1a5a-45ae-95b3-454613387e51-tmp\") pod \"console-b966cd7f8-6pc6x\" (UID: \"a322374a-1a5a-45ae-95b3-454613387e51\") " pod="openshift-console/console-b966cd7f8-6pc6x"
Oct 30 19:21:55.863484 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.863456    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a322374a-1a5a-45ae-95b3-454613387e51-console-config\") pod \"console-b966cd7f8-6pc6x\" (UID: \"a322374a-1a5a-45ae-95b3-454613387e51\") " pod="openshift-console/console-b966cd7f8-6pc6x"
Oct 30 19:21:55.863567 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.863549    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a322374a-1a5a-45ae-95b3-454613387e51-oauth-serving-cert\") pod \"console-b966cd7f8-6pc6x\" (UID: \"a322374a-1a5a-45ae-95b3-454613387e51\") " pod="openshift-console/console-b966cd7f8-6pc6x"
Oct 30 19:21:55.863769 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.863744    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a322374a-1a5a-45ae-95b3-454613387e51-trusted-ca-bundle\") pod \"console-b966cd7f8-6pc6x\" (UID: \"a322374a-1a5a-45ae-95b3-454613387e51\") " pod="openshift-console/console-b966cd7f8-6pc6x"
Oct 30 19:21:55.864194 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.864172    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a322374a-1a5a-45ae-95b3-454613387e51-service-ca\") pod \"console-b966cd7f8-6pc6x\" (UID: \"a322374a-1a5a-45ae-95b3-454613387e51\") " pod="openshift-console/console-b966cd7f8-6pc6x"
Oct 30 19:21:55.866480 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.866456    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a322374a-1a5a-45ae-95b3-454613387e51-console-serving-cert\") pod \"console-b966cd7f8-6pc6x\" (UID: \"a322374a-1a5a-45ae-95b3-454613387e51\") " pod="openshift-console/console-b966cd7f8-6pc6x"
Oct 30 19:21:55.867267 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.867242    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a322374a-1a5a-45ae-95b3-454613387e51-console-oauth-config\") pod \"console-b966cd7f8-6pc6x\" (UID: \"a322374a-1a5a-45ae-95b3-454613387e51\") " pod="openshift-console/console-b966cd7f8-6pc6x"
Oct 30 19:21:55.883839 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:55.883813    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pxxl\" (UniqueName: \"kubernetes.io/projected/a322374a-1a5a-45ae-95b3-454613387e51-kube-api-access-4pxxl\") pod \"console-b966cd7f8-6pc6x\" (UID: \"a322374a-1a5a-45ae-95b3-454613387e51\") " pod="openshift-console/console-b966cd7f8-6pc6x"
Oct 30 19:21:56.088753 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:56.088729    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b966cd7f8-6pc6x"
Oct 30 19:21:56.285089 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:56.285014    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="44b9fc01-6837-46cd-bf01-40cb8fcb90f5" path="/var/lib/kubelet/pods/44b9fc01-6837-46cd-bf01-40cb8fcb90f5/volumes"
Oct 30 19:21:56.418748 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:56.418716    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b966cd7f8-6pc6x"]
Oct 30 19:21:56.421850 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:21:56.421821    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda322374a_1a5a_45ae_95b3_454613387e51.slice/crio-ceccc91259d861fdabac3b35c94b168b620242167123065f649811499137a964 WatchSource:0}: Error finding container ceccc91259d861fdabac3b35c94b168b620242167123065f649811499137a964: Status 404 returned error can't find the container with id ceccc91259d861fdabac3b35c94b168b620242167123065f649811499137a964
Oct 30 19:21:56.509783 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:56.509756    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa815a19-9652-4492-9d04-5aa25725131c","Type":"ContainerStarted","Data":"86f28bedc9659c1e77f4c0ee8ac8edb37169be61ca8e9bc0473822de4c9d5d9d"}
Oct 30 19:21:56.509924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:56.509836    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa815a19-9652-4492-9d04-5aa25725131c","Type":"ContainerStarted","Data":"55258da4c0907d2fe3f83a8c332c6ddbc8485f0e8802afb12131a1a9fefdc691"}
Oct 30 19:21:56.509924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:56.509894    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa815a19-9652-4492-9d04-5aa25725131c","Type":"ContainerStarted","Data":"639d982573688c26aa57bdf2a8080b302f4834739d279ac092a762cddfcf6d81"}
Oct 30 19:21:56.509924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:56.509921    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa815a19-9652-4492-9d04-5aa25725131c","Type":"ContainerStarted","Data":"c6898cbd9ab1475e8f18bd0bbe8f302cdb04b075f607379f90657946a66ab296"}
Oct 30 19:21:56.510054 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:56.509949    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa815a19-9652-4492-9d04-5aa25725131c","Type":"ContainerStarted","Data":"e51a6c04f4fc18d0c2f6a751efa541e159b1d0c5d26b468d4228f0c9a16b1825"}
Oct 30 19:21:56.510054 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:56.509972    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa815a19-9652-4492-9d04-5aa25725131c","Type":"ContainerStarted","Data":"a0307bc021b126a4fd20b8248304eb4e566880aa3a62af69e2a415c335110712"}
Oct 30 19:21:56.511558 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:56.511534    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b966cd7f8-6pc6x" event={"ID":"a322374a-1a5a-45ae-95b3-454613387e51","Type":"ContainerStarted","Data":"33cbcbae70732bb57eadd40e2795a2778678ac4794d474b2f75bfb05fe24f90d"}
Oct 30 19:21:56.511671 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:56.511574    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b966cd7f8-6pc6x" event={"ID":"a322374a-1a5a-45ae-95b3-454613387e51","Type":"ContainerStarted","Data":"ceccc91259d861fdabac3b35c94b168b620242167123065f649811499137a964"}
Oct 30 19:21:56.567249 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:56.567153    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.542183801 podStartE2EDuration="3.567135395s" totalImagesPullingTime="0s" totalInitContainerRuntime="24.951594ms" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:56.566037892 +0000 UTC m=+28.408969831" watchObservedRunningTime="2026-10-30 19:21:56.567135395 +0000 UTC m=+28.410067334"
Oct 30 19:21:57.561320 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:57.561262    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-b966cd7f8-6pc6x" podStartSLOduration=2.561246097 podStartE2EDuration="2.561246097s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:57.560954303 +0000 UTC m=+29.403886244" watchObservedRunningTime="2026-10-30 19:21:57.561246097 +0000 UTC m=+29.404178036"
Oct 30 19:21:58.847537 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:58.847499    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-nw4jx"
Oct 30 19:21:58.992657 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:21:58.992621    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:22:02.061142 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:02.061101    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:22:02.078012 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:02.077982    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:22:02.554265 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:02.554231    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:22:05.700722 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:05.700686    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5j799"
Oct 30 19:22:06.089450 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:06.089355    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-b966cd7f8-6pc6x"
Oct 30 19:22:06.089450 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:06.089428    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-b966cd7f8-6pc6x"
Oct 30 19:22:06.094319 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:06.094296    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-b966cd7f8-6pc6x"
Oct 30 19:22:06.556479 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:06.556449    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-b966cd7f8-6pc6x"
Oct 30 19:22:06.625421 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:06.625390    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bdc9bd9c6-mww4x"]
Oct 30 19:22:17.499407 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:22:17.499362    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:22:30.506688 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:30.506651    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c4d7556c9-lgj6h"]
Oct 30 19:22:30.511469 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:30.511452    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c4d7556c9-lgj6h"
Oct 30 19:22:30.540896 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:30.540846    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c4d7556c9-lgj6h"]
Oct 30 19:22:30.607979 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:30.607942    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/adf839a8-8656-4196-a59f-292012c006f6-trusted-ca-bundle\") pod \"console-6c4d7556c9-lgj6h\" (UID: \"adf839a8-8656-4196-a59f-292012c006f6\") " pod="openshift-console/console-6c4d7556c9-lgj6h"
Oct 30 19:22:30.608174 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:30.608007    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/adf839a8-8656-4196-a59f-292012c006f6-tmp\") pod \"console-6c4d7556c9-lgj6h\" (UID: \"adf839a8-8656-4196-a59f-292012c006f6\") " pod="openshift-console/console-6c4d7556c9-lgj6h"
Oct 30 19:22:30.608174 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:30.608099    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/adf839a8-8656-4196-a59f-292012c006f6-console-config\") pod \"console-6c4d7556c9-lgj6h\" (UID: \"adf839a8-8656-4196-a59f-292012c006f6\") " pod="openshift-console/console-6c4d7556c9-lgj6h"
Oct 30 19:22:30.608174 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:30.608134    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/adf839a8-8656-4196-a59f-292012c006f6-service-ca\") pod \"console-6c4d7556c9-lgj6h\" (UID: \"adf839a8-8656-4196-a59f-292012c006f6\") " pod="openshift-console/console-6c4d7556c9-lgj6h"
Oct 30 19:22:30.608174 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:30.608165    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/adf839a8-8656-4196-a59f-292012c006f6-oauth-serving-cert\") pod \"console-6c4d7556c9-lgj6h\" (UID: \"adf839a8-8656-4196-a59f-292012c006f6\") " pod="openshift-console/console-6c4d7556c9-lgj6h"
Oct 30 19:22:30.608348 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:30.608250    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-strjd\" (UniqueName: \"kubernetes.io/projected/adf839a8-8656-4196-a59f-292012c006f6-kube-api-access-strjd\") pod \"console-6c4d7556c9-lgj6h\" (UID: \"adf839a8-8656-4196-a59f-292012c006f6\") " pod="openshift-console/console-6c4d7556c9-lgj6h"
Oct 30 19:22:30.608348 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:30.608272    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/adf839a8-8656-4196-a59f-292012c006f6-console-serving-cert\") pod \"console-6c4d7556c9-lgj6h\" (UID: \"adf839a8-8656-4196-a59f-292012c006f6\") " pod="openshift-console/console-6c4d7556c9-lgj6h"
Oct 30 19:22:30.608348 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:30.608296    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/adf839a8-8656-4196-a59f-292012c006f6-console-oauth-config\") pod \"console-6c4d7556c9-lgj6h\" (UID: \"adf839a8-8656-4196-a59f-292012c006f6\") " pod="openshift-console/console-6c4d7556c9-lgj6h"
Oct 30 19:22:30.708720 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:30.708681    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/adf839a8-8656-4196-a59f-292012c006f6-trusted-ca-bundle\") pod \"console-6c4d7556c9-lgj6h\" (UID: \"adf839a8-8656-4196-a59f-292012c006f6\") " pod="openshift-console/console-6c4d7556c9-lgj6h"
Oct 30 19:22:30.708938 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:30.708737    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/adf839a8-8656-4196-a59f-292012c006f6-tmp\") pod \"console-6c4d7556c9-lgj6h\" (UID: \"adf839a8-8656-4196-a59f-292012c006f6\") " pod="openshift-console/console-6c4d7556c9-lgj6h"
Oct 30 19:22:30.708938 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:30.708767    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/adf839a8-8656-4196-a59f-292012c006f6-console-config\") pod \"console-6c4d7556c9-lgj6h\" (UID: \"adf839a8-8656-4196-a59f-292012c006f6\") " pod="openshift-console/console-6c4d7556c9-lgj6h"
Oct 30 19:22:30.708938 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:30.708898    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/adf839a8-8656-4196-a59f-292012c006f6-service-ca\") pod \"console-6c4d7556c9-lgj6h\" (UID: \"adf839a8-8656-4196-a59f-292012c006f6\") " pod="openshift-console/console-6c4d7556c9-lgj6h"
Oct 30 19:22:30.709070 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:30.708956    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/adf839a8-8656-4196-a59f-292012c006f6-oauth-serving-cert\") pod \"console-6c4d7556c9-lgj6h\" (UID: \"adf839a8-8656-4196-a59f-292012c006f6\") " pod="openshift-console/console-6c4d7556c9-lgj6h"
Oct 30 19:22:30.709070 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:30.708999    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-strjd\" (UniqueName: \"kubernetes.io/projected/adf839a8-8656-4196-a59f-292012c006f6-kube-api-access-strjd\") pod \"console-6c4d7556c9-lgj6h\" (UID: \"adf839a8-8656-4196-a59f-292012c006f6\") " pod="openshift-console/console-6c4d7556c9-lgj6h"
Oct 30 19:22:30.709070 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:30.709023    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/adf839a8-8656-4196-a59f-292012c006f6-console-serving-cert\") pod \"console-6c4d7556c9-lgj6h\" (UID: \"adf839a8-8656-4196-a59f-292012c006f6\") " pod="openshift-console/console-6c4d7556c9-lgj6h"
Oct 30 19:22:30.709070 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:30.709066    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/adf839a8-8656-4196-a59f-292012c006f6-console-oauth-config\") pod \"console-6c4d7556c9-lgj6h\" (UID: \"adf839a8-8656-4196-a59f-292012c006f6\") " pod="openshift-console/console-6c4d7556c9-lgj6h"
Oct 30 19:22:30.709266 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:30.709248    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/adf839a8-8656-4196-a59f-292012c006f6-tmp\") pod \"console-6c4d7556c9-lgj6h\" (UID: \"adf839a8-8656-4196-a59f-292012c006f6\") " pod="openshift-console/console-6c4d7556c9-lgj6h"
Oct 30 19:22:30.709697 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:30.709673    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/adf839a8-8656-4196-a59f-292012c006f6-console-config\") pod \"console-6c4d7556c9-lgj6h\" (UID: \"adf839a8-8656-4196-a59f-292012c006f6\") " pod="openshift-console/console-6c4d7556c9-lgj6h"
Oct 30 19:22:30.709828 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:30.709675    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/adf839a8-8656-4196-a59f-292012c006f6-service-ca\") pod \"console-6c4d7556c9-lgj6h\" (UID: \"adf839a8-8656-4196-a59f-292012c006f6\") " pod="openshift-console/console-6c4d7556c9-lgj6h"
Oct 30 19:22:30.709934 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:30.709754    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/adf839a8-8656-4196-a59f-292012c006f6-oauth-serving-cert\") pod \"console-6c4d7556c9-lgj6h\" (UID: \"adf839a8-8656-4196-a59f-292012c006f6\") " pod="openshift-console/console-6c4d7556c9-lgj6h"
Oct 30 19:22:30.709996 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:30.709927    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/adf839a8-8656-4196-a59f-292012c006f6-trusted-ca-bundle\") pod \"console-6c4d7556c9-lgj6h\" (UID: \"adf839a8-8656-4196-a59f-292012c006f6\") " pod="openshift-console/console-6c4d7556c9-lgj6h"
Oct 30 19:22:30.711821 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:30.711797    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/adf839a8-8656-4196-a59f-292012c006f6-console-oauth-config\") pod \"console-6c4d7556c9-lgj6h\" (UID: \"adf839a8-8656-4196-a59f-292012c006f6\") " pod="openshift-console/console-6c4d7556c9-lgj6h"
Oct 30 19:22:30.711965 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:30.711942    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/adf839a8-8656-4196-a59f-292012c006f6-console-serving-cert\") pod \"console-6c4d7556c9-lgj6h\" (UID: \"adf839a8-8656-4196-a59f-292012c006f6\") " pod="openshift-console/console-6c4d7556c9-lgj6h"
Oct 30 19:22:30.731282 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:30.731253    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-strjd\" (UniqueName: \"kubernetes.io/projected/adf839a8-8656-4196-a59f-292012c006f6-kube-api-access-strjd\") pod \"console-6c4d7556c9-lgj6h\" (UID: \"adf839a8-8656-4196-a59f-292012c006f6\") " pod="openshift-console/console-6c4d7556c9-lgj6h"
Oct 30 19:22:30.823131 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:30.823050    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c4d7556c9-lgj6h"
Oct 30 19:22:31.130446 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:31.130415    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c4d7556c9-lgj6h"]
Oct 30 19:22:31.133569 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:22:31.133541    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadf839a8_8656_4196_a59f_292012c006f6.slice/crio-6fca31ab3ea0b5d7b0f0770add2bec398caeafc9b0378a136f0ddcdd185aa6b5 WatchSource:0}: Error finding container 6fca31ab3ea0b5d7b0f0770add2bec398caeafc9b0378a136f0ddcdd185aa6b5: Status 404 returned error can't find the container with id 6fca31ab3ea0b5d7b0f0770add2bec398caeafc9b0378a136f0ddcdd185aa6b5
Oct 30 19:22:31.646907 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:31.646839    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-6bdc9bd9c6-mww4x" podUID="55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56" containerName="console" containerID="cri-o://207c492fbd2114ccdc9ec9124790e3531fba903a26c4a742b171c4fd7978e694" gracePeriod=15
Oct 30 19:22:31.663076 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:31.663049    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c4d7556c9-lgj6h" event={"ID":"adf839a8-8656-4196-a59f-292012c006f6","Type":"ContainerStarted","Data":"de37cc62301b83585cabab7a96731fe79e86bc5307d2de61bfffea7df416d1ef"}
Oct 30 19:22:31.663202 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:31.663101    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c4d7556c9-lgj6h" event={"ID":"adf839a8-8656-4196-a59f-292012c006f6","Type":"ContainerStarted","Data":"6fca31ab3ea0b5d7b0f0770add2bec398caeafc9b0378a136f0ddcdd185aa6b5"}
Oct 30 19:22:31.722761 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:31.722715    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-6c4d7556c9-lgj6h" podStartSLOduration=1.722702754 podStartE2EDuration="1.722702754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:22:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:22:31.708772875 +0000 UTC m=+63.551704815" watchObservedRunningTime="2026-10-30 19:22:31.722702754 +0000 UTC m=+63.565634693"
Oct 30 19:22:31.898118 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:31.898058    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:22:32.021662 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:32.021622    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-oauth-serving-cert\") pod \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") "
Oct 30 19:22:32.021887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:32.021688    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-service-ca\" (UniqueName: \"kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-service-ca\") pod \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") "
Oct 30 19:22:32.021887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:32.021714    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-console-config\" (UniqueName: \"kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-console-config\") pod \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") "
Oct 30 19:22:32.021887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:32.021757    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-console-serving-cert\") pod \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") "
Oct 30 19:22:32.021887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:32.021796    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-tmp\" (UniqueName: \"kubernetes.io/empty-dir/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-tmp\") pod \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") "
Oct 30 19:22:32.021887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:32.021849    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-kube-api-access-vv49j\" (UniqueName: \"kubernetes.io/projected/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-kube-api-access-vv49j\") pod \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") "
Oct 30 19:22:32.022134 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:32.021920    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-console-oauth-config\") pod \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") "
Oct 30 19:22:32.022134 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:32.021980    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-trusted-ca-bundle\") pod \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\" (UID: \"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56\") "
Oct 30 19:22:32.022134 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:32.022113    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-oauth-serving-cert" pod "55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56" (UID: "55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:22:32.022317 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:32.022294    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-tmp" pod "55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56" (UID: "55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:22:32.022374 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:32.022292    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-service-ca" pod "55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56" (UID: "55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:22:32.022423 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:32.022388    5907 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-service-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:22:32.022423 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:32.022409    5907 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-tmp\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:22:32.022501 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:32.022424    5907 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-oauth-serving-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:22:32.022501 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:32.022490    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-console-config" pod "55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56" (UID: "55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:22:32.022566 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:32.022549    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-trusted-ca-bundle" pod "55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56" (UID: "55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:22:32.024412 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:32.024384    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-kube-api-access-vv49j" pod "55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56" (UID: "55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56"). InnerVolumeSpecName "kube-api-access-vv49j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:22:32.024412 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:32.024392    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-console-oauth-config" pod "55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56" (UID: "55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:22:32.024557 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:32.024456    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-console-serving-cert" pod "55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56" (UID: "55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:22:32.122892 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:32.122829    5907 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-trusted-ca-bundle\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:22:32.122892 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:32.122891    5907 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-console-config\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:22:32.123097 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:32.122906    5907 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-console-serving-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:22:32.123097 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:32.122919    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vv49j\" (UniqueName: \"kubernetes.io/projected/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-kube-api-access-vv49j\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:22:32.123097 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:32.122932    5907 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56-console-oauth-config\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:22:32.668415 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:32.668380    5907 generic.go:356] "Generic (PLEG): container finished" podID="55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56" containerID="207c492fbd2114ccdc9ec9124790e3531fba903a26c4a742b171c4fd7978e694" exitCode=0
Oct 30 19:22:32.668880 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:32.668428    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bdc9bd9c6-mww4x"
Oct 30 19:22:32.668880 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:32.668461    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bdc9bd9c6-mww4x" event={"ID":"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56","Type":"ContainerDied","Data":"207c492fbd2114ccdc9ec9124790e3531fba903a26c4a742b171c4fd7978e694"}
Oct 30 19:22:32.668880 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:32.668516    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bdc9bd9c6-mww4x" event={"ID":"55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56","Type":"ContainerDied","Data":"bebff447d4392a1b68f1a0d6050ed163c27381784c1b08ee6f917e60a37309f0"}
Oct 30 19:22:32.668880 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:32.668536    5907 scope.go:122] "RemoveContainer" containerID="207c492fbd2114ccdc9ec9124790e3531fba903a26c4a742b171c4fd7978e694"
Oct 30 19:22:32.676731 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:32.676710    5907 scope.go:122] "RemoveContainer" containerID="207c492fbd2114ccdc9ec9124790e3531fba903a26c4a742b171c4fd7978e694"
Oct 30 19:22:32.677014 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:22:32.676997    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"207c492fbd2114ccdc9ec9124790e3531fba903a26c4a742b171c4fd7978e694\": container with ID starting with 207c492fbd2114ccdc9ec9124790e3531fba903a26c4a742b171c4fd7978e694 not found: ID does not exist" containerID="207c492fbd2114ccdc9ec9124790e3531fba903a26c4a742b171c4fd7978e694"
Oct 30 19:22:32.677064 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:32.677023    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"207c492fbd2114ccdc9ec9124790e3531fba903a26c4a742b171c4fd7978e694"} err="failed to get container status \"207c492fbd2114ccdc9ec9124790e3531fba903a26c4a742b171c4fd7978e694\": rpc error: code = NotFound desc = could not find container \"207c492fbd2114ccdc9ec9124790e3531fba903a26c4a742b171c4fd7978e694\": container with ID starting with 207c492fbd2114ccdc9ec9124790e3531fba903a26c4a742b171c4fd7978e694 not found: ID does not exist"
Oct 30 19:22:32.724799 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:32.724769    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bdc9bd9c6-mww4x"]
Oct 30 19:22:32.738501 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:32.738472    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6bdc9bd9c6-mww4x"]
Oct 30 19:22:34.285750 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:34.285702    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56" path="/var/lib/kubelet/pods/55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56/volumes"
Oct 30 19:22:40.823715 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:40.823665    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c4d7556c9-lgj6h"
Oct 30 19:22:40.824180 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:40.823740    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c4d7556c9-lgj6h"
Oct 30 19:22:40.828616 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:40.828593    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c4d7556c9-lgj6h"
Oct 30 19:22:41.707079 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:41.707048    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c4d7556c9-lgj6h"
Oct 30 19:22:41.783475 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:22:41.783446    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b966cd7f8-6pc6x"]
Oct 30 19:22:47.498850 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:22:47.498812    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:23:06.803694 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:06.803627    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-b966cd7f8-6pc6x" podUID="a322374a-1a5a-45ae-95b3-454613387e51" containerName="console" containerID="cri-o://33cbcbae70732bb57eadd40e2795a2778678ac4794d474b2f75bfb05fe24f90d" gracePeriod=15
Oct 30 19:23:07.048120 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:07.048089    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b966cd7f8-6pc6x"
Oct 30 19:23:07.142416 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:07.142379    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a322374a-1a5a-45ae-95b3-454613387e51-kube-api-access-4pxxl\" (UniqueName: \"kubernetes.io/projected/a322374a-1a5a-45ae-95b3-454613387e51-kube-api-access-4pxxl\") pod \"a322374a-1a5a-45ae-95b3-454613387e51\" (UID: \"a322374a-1a5a-45ae-95b3-454613387e51\") "
Oct 30 19:23:07.142416 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:07.142418    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a322374a-1a5a-45ae-95b3-454613387e51-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a322374a-1a5a-45ae-95b3-454613387e51-console-serving-cert\") pod \"a322374a-1a5a-45ae-95b3-454613387e51\" (UID: \"a322374a-1a5a-45ae-95b3-454613387e51\") "
Oct 30 19:23:07.142643 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:07.142437    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a322374a-1a5a-45ae-95b3-454613387e51-service-ca\" (UniqueName: \"kubernetes.io/configmap/a322374a-1a5a-45ae-95b3-454613387e51-service-ca\") pod \"a322374a-1a5a-45ae-95b3-454613387e51\" (UID: \"a322374a-1a5a-45ae-95b3-454613387e51\") "
Oct 30 19:23:07.142643 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:07.142504    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a322374a-1a5a-45ae-95b3-454613387e51-tmp\" (UniqueName: \"kubernetes.io/empty-dir/a322374a-1a5a-45ae-95b3-454613387e51-tmp\") pod \"a322374a-1a5a-45ae-95b3-454613387e51\" (UID: \"a322374a-1a5a-45ae-95b3-454613387e51\") "
Oct 30 19:23:07.142643 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:07.142557    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a322374a-1a5a-45ae-95b3-454613387e51-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a322374a-1a5a-45ae-95b3-454613387e51-trusted-ca-bundle\") pod \"a322374a-1a5a-45ae-95b3-454613387e51\" (UID: \"a322374a-1a5a-45ae-95b3-454613387e51\") "
Oct 30 19:23:07.142643 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:07.142594    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a322374a-1a5a-45ae-95b3-454613387e51-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a322374a-1a5a-45ae-95b3-454613387e51-oauth-serving-cert\") pod \"a322374a-1a5a-45ae-95b3-454613387e51\" (UID: \"a322374a-1a5a-45ae-95b3-454613387e51\") "
Oct 30 19:23:07.142643 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:07.142625    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a322374a-1a5a-45ae-95b3-454613387e51-console-config\" (UniqueName: \"kubernetes.io/configmap/a322374a-1a5a-45ae-95b3-454613387e51-console-config\") pod \"a322374a-1a5a-45ae-95b3-454613387e51\" (UID: \"a322374a-1a5a-45ae-95b3-454613387e51\") "
Oct 30 19:23:07.142917 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:07.142660    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a322374a-1a5a-45ae-95b3-454613387e51-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a322374a-1a5a-45ae-95b3-454613387e51-console-oauth-config\") pod \"a322374a-1a5a-45ae-95b3-454613387e51\" (UID: \"a322374a-1a5a-45ae-95b3-454613387e51\") "
Oct 30 19:23:07.143042 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:07.142918    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a322374a-1a5a-45ae-95b3-454613387e51-tmp" pod "a322374a-1a5a-45ae-95b3-454613387e51" (UID: "a322374a-1a5a-45ae-95b3-454613387e51"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:23:07.143042 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:07.143001    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a322374a-1a5a-45ae-95b3-454613387e51-service-ca" pod "a322374a-1a5a-45ae-95b3-454613387e51" (UID: "a322374a-1a5a-45ae-95b3-454613387e51"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:23:07.143151 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:07.143078    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a322374a-1a5a-45ae-95b3-454613387e51-oauth-serving-cert" pod "a322374a-1a5a-45ae-95b3-454613387e51" (UID: "a322374a-1a5a-45ae-95b3-454613387e51"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:23:07.143151 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:07.143096    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a322374a-1a5a-45ae-95b3-454613387e51-trusted-ca-bundle" pod "a322374a-1a5a-45ae-95b3-454613387e51" (UID: "a322374a-1a5a-45ae-95b3-454613387e51"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:23:07.143352 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:07.143312    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a322374a-1a5a-45ae-95b3-454613387e51-console-config" pod "a322374a-1a5a-45ae-95b3-454613387e51" (UID: "a322374a-1a5a-45ae-95b3-454613387e51"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:23:07.144807 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:07.144782    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a322374a-1a5a-45ae-95b3-454613387e51-console-serving-cert" pod "a322374a-1a5a-45ae-95b3-454613387e51" (UID: "a322374a-1a5a-45ae-95b3-454613387e51"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:23:07.144807 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:07.144796    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a322374a-1a5a-45ae-95b3-454613387e51-kube-api-access-4pxxl" pod "a322374a-1a5a-45ae-95b3-454613387e51" (UID: "a322374a-1a5a-45ae-95b3-454613387e51"). InnerVolumeSpecName "kube-api-access-4pxxl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:23:07.145120 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:07.145097    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a322374a-1a5a-45ae-95b3-454613387e51-console-oauth-config" pod "a322374a-1a5a-45ae-95b3-454613387e51" (UID: "a322374a-1a5a-45ae-95b3-454613387e51"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:23:07.244083 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:07.244051    5907 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a322374a-1a5a-45ae-95b3-454613387e51-console-oauth-config\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:23:07.244083 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:07.244078    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4pxxl\" (UniqueName: \"kubernetes.io/projected/a322374a-1a5a-45ae-95b3-454613387e51-kube-api-access-4pxxl\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:23:07.244083 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:07.244087    5907 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a322374a-1a5a-45ae-95b3-454613387e51-console-serving-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:23:07.244316 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:07.244096    5907 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a322374a-1a5a-45ae-95b3-454613387e51-service-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:23:07.244316 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:07.244104    5907 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a322374a-1a5a-45ae-95b3-454613387e51-tmp\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:23:07.244316 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:07.244112    5907 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a322374a-1a5a-45ae-95b3-454613387e51-trusted-ca-bundle\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:23:07.244316 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:07.244120    5907 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a322374a-1a5a-45ae-95b3-454613387e51-oauth-serving-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:23:07.244316 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:07.244128    5907 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a322374a-1a5a-45ae-95b3-454613387e51-console-config\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:23:07.810068 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:07.810028    5907 generic.go:356] "Generic (PLEG): container finished" podID="a322374a-1a5a-45ae-95b3-454613387e51" containerID="33cbcbae70732bb57eadd40e2795a2778678ac4794d474b2f75bfb05fe24f90d" exitCode=0
Oct 30 19:23:07.810718 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:07.810089    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b966cd7f8-6pc6x"
Oct 30 19:23:07.810718 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:07.810087    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b966cd7f8-6pc6x" event={"ID":"a322374a-1a5a-45ae-95b3-454613387e51","Type":"ContainerDied","Data":"33cbcbae70732bb57eadd40e2795a2778678ac4794d474b2f75bfb05fe24f90d"}
Oct 30 19:23:07.810718 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:07.810204    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b966cd7f8-6pc6x" event={"ID":"a322374a-1a5a-45ae-95b3-454613387e51","Type":"ContainerDied","Data":"ceccc91259d861fdabac3b35c94b168b620242167123065f649811499137a964"}
Oct 30 19:23:07.810718 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:07.810229    5907 scope.go:122] "RemoveContainer" containerID="33cbcbae70732bb57eadd40e2795a2778678ac4794d474b2f75bfb05fe24f90d"
Oct 30 19:23:07.819561 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:07.819541    5907 scope.go:122] "RemoveContainer" containerID="33cbcbae70732bb57eadd40e2795a2778678ac4794d474b2f75bfb05fe24f90d"
Oct 30 19:23:07.819836 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:23:07.819819    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33cbcbae70732bb57eadd40e2795a2778678ac4794d474b2f75bfb05fe24f90d\": container with ID starting with 33cbcbae70732bb57eadd40e2795a2778678ac4794d474b2f75bfb05fe24f90d not found: ID does not exist" containerID="33cbcbae70732bb57eadd40e2795a2778678ac4794d474b2f75bfb05fe24f90d"
Oct 30 19:23:07.819900 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:07.819844    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33cbcbae70732bb57eadd40e2795a2778678ac4794d474b2f75bfb05fe24f90d"} err="failed to get container status \"33cbcbae70732bb57eadd40e2795a2778678ac4794d474b2f75bfb05fe24f90d\": rpc error: code = NotFound desc = could not find container \"33cbcbae70732bb57eadd40e2795a2778678ac4794d474b2f75bfb05fe24f90d\": container with ID starting with 33cbcbae70732bb57eadd40e2795a2778678ac4794d474b2f75bfb05fe24f90d not found: ID does not exist"
Oct 30 19:23:07.865192 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:07.865160    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b966cd7f8-6pc6x"]
Oct 30 19:23:07.878684 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:07.878599    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-b966cd7f8-6pc6x"]
Oct 30 19:23:08.284804 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:23:08.284727    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a322374a-1a5a-45ae-95b3-454613387e51" path="/var/lib/kubelet/pods/a322374a-1a5a-45ae-95b3-454613387e51/volumes"
Oct 30 19:23:17.498720 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:23:17.498671    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:23:47.498180 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:23:47.498137    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:24:17.498829 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:24:17.498788    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:24:47.498263 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:24:47.498224    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:25:17.498699 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:25:17.498663    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:25:32.519206 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:32.519169    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-5d58dddd7d-4txzv"]
Oct 30 19:25:32.519616 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:32.519513    5907 cpu_manager.go:414] "containerMap: removing container" podUID="a322374a-1a5a-45ae-95b3-454613387e51" containerName="console"
Oct 30 19:25:32.519616 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:32.519525    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a322374a-1a5a-45ae-95b3-454613387e51" containerName="console"
Oct 30 19:25:32.519616 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:32.519549    5907 cpu_manager.go:414] "containerMap: removing container" podUID="55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56" containerName="console"
Oct 30 19:25:32.519616 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:32.519554    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56" containerName="console"
Oct 30 19:25:32.519616 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:32.519611    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="a322374a-1a5a-45ae-95b3-454613387e51" containerName="console"
Oct 30 19:25:32.519770 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:32.519622    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="55ab4c0b-7882-4bb1-ab3c-75cc7af5ce56" containerName="console"
Oct 30 19:25:32.522776 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:32.522760    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5d58dddd7d-4txzv"
Oct 30 19:25:32.588445 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:32.588412    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-5d58dddd7d-4txzv"]
Oct 30 19:25:32.617768 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:32.617730    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z7v2\" (UniqueName: \"kubernetes.io/projected/2d19dee6-c164-4706-a7c4-35edcb64eafa-kube-api-access-4z7v2\") pod \"opendatahub-operator-controller-manager-5d58dddd7d-4txzv\" (UID: \"2d19dee6-c164-4706-a7c4-35edcb64eafa\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5d58dddd7d-4txzv"
Oct 30 19:25:32.617974 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:32.617785    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2d19dee6-c164-4706-a7c4-35edcb64eafa-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d58dddd7d-4txzv\" (UID: \"2d19dee6-c164-4706-a7c4-35edcb64eafa\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5d58dddd7d-4txzv"
Oct 30 19:25:32.617974 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:32.617818    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2d19dee6-c164-4706-a7c4-35edcb64eafa-manifests\") pod \"opendatahub-operator-controller-manager-5d58dddd7d-4txzv\" (UID: \"2d19dee6-c164-4706-a7c4-35edcb64eafa\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5d58dddd7d-4txzv"
Oct 30 19:25:32.617974 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:32.617911    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2d19dee6-c164-4706-a7c4-35edcb64eafa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d58dddd7d-4txzv\" (UID: \"2d19dee6-c164-4706-a7c4-35edcb64eafa\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5d58dddd7d-4txzv"
Oct 30 19:25:32.618082 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:32.618012    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d19dee6-c164-4706-a7c4-35edcb64eafa-tmp\") pod \"opendatahub-operator-controller-manager-5d58dddd7d-4txzv\" (UID: \"2d19dee6-c164-4706-a7c4-35edcb64eafa\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5d58dddd7d-4txzv"
Oct 30 19:25:32.618135 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:32.618116    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2d19dee6-c164-4706-a7c4-35edcb64eafa-metrics-cert\") pod \"opendatahub-operator-controller-manager-5d58dddd7d-4txzv\" (UID: \"2d19dee6-c164-4706-a7c4-35edcb64eafa\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5d58dddd7d-4txzv"
Oct 30 19:25:32.620669 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:32.620652    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 30 19:25:32.621104 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:32.621084    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 30 19:25:32.621647 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:32.621630    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Oct 30 19:25:32.621758 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:32.621746    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-8hq8l\""
Oct 30 19:25:32.622286 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:32.622272    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Oct 30 19:25:32.622361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:32.622291    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Oct 30 19:25:32.718699 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:32.718664    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z7v2\" (UniqueName: \"kubernetes.io/projected/2d19dee6-c164-4706-a7c4-35edcb64eafa-kube-api-access-4z7v2\") pod \"opendatahub-operator-controller-manager-5d58dddd7d-4txzv\" (UID: \"2d19dee6-c164-4706-a7c4-35edcb64eafa\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5d58dddd7d-4txzv"
Oct 30 19:25:32.718699 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:32.718702    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2d19dee6-c164-4706-a7c4-35edcb64eafa-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d58dddd7d-4txzv\" (UID: \"2d19dee6-c164-4706-a7c4-35edcb64eafa\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5d58dddd7d-4txzv"
Oct 30 19:25:32.718984 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:32.718736    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2d19dee6-c164-4706-a7c4-35edcb64eafa-manifests\") pod \"opendatahub-operator-controller-manager-5d58dddd7d-4txzv\" (UID: \"2d19dee6-c164-4706-a7c4-35edcb64eafa\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5d58dddd7d-4txzv"
Oct 30 19:25:32.718984 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:32.718755    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2d19dee6-c164-4706-a7c4-35edcb64eafa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d58dddd7d-4txzv\" (UID: \"2d19dee6-c164-4706-a7c4-35edcb64eafa\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5d58dddd7d-4txzv"
Oct 30 19:25:32.718984 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:32.718846    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d19dee6-c164-4706-a7c4-35edcb64eafa-tmp\") pod \"opendatahub-operator-controller-manager-5d58dddd7d-4txzv\" (UID: \"2d19dee6-c164-4706-a7c4-35edcb64eafa\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5d58dddd7d-4txzv"
Oct 30 19:25:32.719138 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:32.718996    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2d19dee6-c164-4706-a7c4-35edcb64eafa-metrics-cert\") pod \"opendatahub-operator-controller-manager-5d58dddd7d-4txzv\" (UID: \"2d19dee6-c164-4706-a7c4-35edcb64eafa\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5d58dddd7d-4txzv"
Oct 30 19:25:32.719267 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:32.719236    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2d19dee6-c164-4706-a7c4-35edcb64eafa-manifests\") pod \"opendatahub-operator-controller-manager-5d58dddd7d-4txzv\" (UID: \"2d19dee6-c164-4706-a7c4-35edcb64eafa\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5d58dddd7d-4txzv"
Oct 30 19:25:32.721577 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:32.721553    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2d19dee6-c164-4706-a7c4-35edcb64eafa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d58dddd7d-4txzv\" (UID: \"2d19dee6-c164-4706-a7c4-35edcb64eafa\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5d58dddd7d-4txzv"
Oct 30 19:25:32.721577 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:32.721553    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2d19dee6-c164-4706-a7c4-35edcb64eafa-metrics-cert\") pod \"opendatahub-operator-controller-manager-5d58dddd7d-4txzv\" (UID: \"2d19dee6-c164-4706-a7c4-35edcb64eafa\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5d58dddd7d-4txzv"
Oct 30 19:25:32.721710 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:32.721554    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d19dee6-c164-4706-a7c4-35edcb64eafa-tmp\") pod \"opendatahub-operator-controller-manager-5d58dddd7d-4txzv\" (UID: \"2d19dee6-c164-4706-a7c4-35edcb64eafa\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5d58dddd7d-4txzv"
Oct 30 19:25:32.721775 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:32.721758    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2d19dee6-c164-4706-a7c4-35edcb64eafa-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d58dddd7d-4txzv\" (UID: \"2d19dee6-c164-4706-a7c4-35edcb64eafa\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5d58dddd7d-4txzv"
Oct 30 19:25:32.743675 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:32.743639    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z7v2\" (UniqueName: \"kubernetes.io/projected/2d19dee6-c164-4706-a7c4-35edcb64eafa-kube-api-access-4z7v2\") pod \"opendatahub-operator-controller-manager-5d58dddd7d-4txzv\" (UID: \"2d19dee6-c164-4706-a7c4-35edcb64eafa\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5d58dddd7d-4txzv"
Oct 30 19:25:32.837797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:32.837703    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5d58dddd7d-4txzv"
Oct 30 19:25:33.238744 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:33.238707    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-5d58dddd7d-4txzv"]
Oct 30 19:25:33.245543 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:25:33.245515    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d19dee6_c164_4706_a7c4_35edcb64eafa.slice/crio-8e47795ce14cda312c3ff1409d6d865a17534983cc96c0c975db32c872701eee WatchSource:0}: Error finding container 8e47795ce14cda312c3ff1409d6d865a17534983cc96c0c975db32c872701eee: Status 404 returned error can't find the container with id 8e47795ce14cda312c3ff1409d6d865a17534983cc96c0c975db32c872701eee
Oct 30 19:25:33.361708 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:33.361674    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5d58dddd7d-4txzv" event={"ID":"2d19dee6-c164-4706-a7c4-35edcb64eafa","Type":"ContainerStarted","Data":"8e47795ce14cda312c3ff1409d6d865a17534983cc96c0c975db32c872701eee"}
Oct 30 19:25:36.376997 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:36.376955    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5d58dddd7d-4txzv" event={"ID":"2d19dee6-c164-4706-a7c4-35edcb64eafa","Type":"ContainerStarted","Data":"8705976a9b565f8ef50c9a246a85d554325f56245b94162e18f23f35601767cb"}
Oct 30 19:25:37.381944 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:37.381907    5907 generic.go:356] "Generic (PLEG): container finished" podID="2d19dee6-c164-4706-a7c4-35edcb64eafa" containerID="8705976a9b565f8ef50c9a246a85d554325f56245b94162e18f23f35601767cb" exitCode=0
Oct 30 19:25:37.382347 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:37.381997    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5d58dddd7d-4txzv" event={"ID":"2d19dee6-c164-4706-a7c4-35edcb64eafa","Type":"ContainerDied","Data":"8705976a9b565f8ef50c9a246a85d554325f56245b94162e18f23f35601767cb"}
Oct 30 19:25:38.388126 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:38.388088    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5d58dddd7d-4txzv" event={"ID":"2d19dee6-c164-4706-a7c4-35edcb64eafa","Type":"ContainerStarted","Data":"c1185bf85123ebbb18268b032e83a86987905a34784111eaf9c7cd0b79988ba1"}
Oct 30 19:25:38.388629 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:38.388226    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5d58dddd7d-4txzv"
Oct 30 19:25:38.475028 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:38.474982    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5d58dddd7d-4txzv" podStartSLOduration=2.831479898 podStartE2EDuration="6.474968952s" totalImagesPullingTime="3.161837612s" totalInitContainerRuntime="481.651442ms" isStatefulPod=false podCreationTimestamp="2026-10-30 19:25:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:25:38.460965793 +0000 UTC m=+250.303897732" watchObservedRunningTime="2026-10-30 19:25:38.474968952 +0000 UTC m=+250.317900885"
Oct 30 19:25:45.720794 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:45.720757    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-6sdjf"]
Oct 30 19:25:45.729300 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:45.729275    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-6sdjf"
Oct 30 19:25:45.753570 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:45.753537    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\""
Oct 30 19:25:45.753745 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:45.753593    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\""
Oct 30 19:25:45.768425 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:45.768385    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-6sdjf"]
Oct 30 19:25:45.787790 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:45.787762    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-operator-dockercfg-zj6zh\""
Oct 30 19:25:45.838258 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:45.838219    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c523ec8f-2430-498c-9a05-f7d5bf0a0a87-tmp\") pod \"jobset-operator-674cd59dc5-6sdjf\" (UID: \"c523ec8f-2430-498c-9a05-f7d5bf0a0a87\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-6sdjf"
Oct 30 19:25:45.838428 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:45.838284    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-244mp\" (UniqueName: \"kubernetes.io/projected/c523ec8f-2430-498c-9a05-f7d5bf0a0a87-kube-api-access-244mp\") pod \"jobset-operator-674cd59dc5-6sdjf\" (UID: \"c523ec8f-2430-498c-9a05-f7d5bf0a0a87\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-6sdjf"
Oct 30 19:25:45.939554 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:45.939505    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c523ec8f-2430-498c-9a05-f7d5bf0a0a87-tmp\") pod \"jobset-operator-674cd59dc5-6sdjf\" (UID: \"c523ec8f-2430-498c-9a05-f7d5bf0a0a87\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-6sdjf"
Oct 30 19:25:45.939733 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:45.939585    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-244mp\" (UniqueName: \"kubernetes.io/projected/c523ec8f-2430-498c-9a05-f7d5bf0a0a87-kube-api-access-244mp\") pod \"jobset-operator-674cd59dc5-6sdjf\" (UID: \"c523ec8f-2430-498c-9a05-f7d5bf0a0a87\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-6sdjf"
Oct 30 19:25:45.939941 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:45.939920    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c523ec8f-2430-498c-9a05-f7d5bf0a0a87-tmp\") pod \"jobset-operator-674cd59dc5-6sdjf\" (UID: \"c523ec8f-2430-498c-9a05-f7d5bf0a0a87\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-6sdjf"
Oct 30 19:25:45.962735 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:45.962701    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-244mp\" (UniqueName: \"kubernetes.io/projected/c523ec8f-2430-498c-9a05-f7d5bf0a0a87-kube-api-access-244mp\") pod \"jobset-operator-674cd59dc5-6sdjf\" (UID: \"c523ec8f-2430-498c-9a05-f7d5bf0a0a87\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-6sdjf"
Oct 30 19:25:46.059486 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:46.059400    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-6sdjf"
Oct 30 19:25:46.361822 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:46.361794    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-6sdjf"]
Oct 30 19:25:46.364510 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:25:46.364480    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc523ec8f_2430_498c_9a05_f7d5bf0a0a87.slice/crio-265cb3d60e601a9dc42e7d6f84576086294395b213609e6c95fdba9647697373 WatchSource:0}: Error finding container 265cb3d60e601a9dc42e7d6f84576086294395b213609e6c95fdba9647697373: Status 404 returned error can't find the container with id 265cb3d60e601a9dc42e7d6f84576086294395b213609e6c95fdba9647697373
Oct 30 19:25:46.418949 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:46.418916    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-6sdjf" event={"ID":"c523ec8f-2430-498c-9a05-f7d5bf0a0a87","Type":"ContainerStarted","Data":"265cb3d60e601a9dc42e7d6f84576086294395b213609e6c95fdba9647697373"}
Oct 30 19:25:47.509501 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:25:47.509449    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:25:48.540974 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:48.540940    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-dhf4k"]
Oct 30 19:25:48.545291 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:48.545265    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-dhf4k"
Oct 30 19:25:48.563806 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:48.563778    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-dhf4k"]
Oct 30 19:25:48.576414 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:48.576386    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-q9xmr\""
Oct 30 19:25:48.576570 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:48.576421    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Oct 30 19:25:48.576570 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:48.576445    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Oct 30 19:25:48.667278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:48.667243    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6x522\" (UniqueName: \"kubernetes.io/projected/8adead5a-6b26-4f4f-8d0e-c0a38120ca3b-kube-api-access-6x522\") pod \"cert-manager-cainjector-5c57bbd4fc-dhf4k\" (UID: \"8adead5a-6b26-4f4f-8d0e-c0a38120ca3b\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-dhf4k"
Oct 30 19:25:48.667537 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:48.667296    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8adead5a-6b26-4f4f-8d0e-c0a38120ca3b-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-dhf4k\" (UID: \"8adead5a-6b26-4f4f-8d0e-c0a38120ca3b\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-dhf4k"
Oct 30 19:25:48.768448 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:48.768407    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8adead5a-6b26-4f4f-8d0e-c0a38120ca3b-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-dhf4k\" (UID: \"8adead5a-6b26-4f4f-8d0e-c0a38120ca3b\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-dhf4k"
Oct 30 19:25:48.768648 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:48.768555    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6x522\" (UniqueName: \"kubernetes.io/projected/8adead5a-6b26-4f4f-8d0e-c0a38120ca3b-kube-api-access-6x522\") pod \"cert-manager-cainjector-5c57bbd4fc-dhf4k\" (UID: \"8adead5a-6b26-4f4f-8d0e-c0a38120ca3b\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-dhf4k"
Oct 30 19:25:48.793142 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:48.793068    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8adead5a-6b26-4f4f-8d0e-c0a38120ca3b-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-dhf4k\" (UID: \"8adead5a-6b26-4f4f-8d0e-c0a38120ca3b\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-dhf4k"
Oct 30 19:25:48.793271 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:48.793143    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6x522\" (UniqueName: \"kubernetes.io/projected/8adead5a-6b26-4f4f-8d0e-c0a38120ca3b-kube-api-access-6x522\") pod \"cert-manager-cainjector-5c57bbd4fc-dhf4k\" (UID: \"8adead5a-6b26-4f4f-8d0e-c0a38120ca3b\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-dhf4k"
Oct 30 19:25:48.858808 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:48.858774    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-dhf4k"
Oct 30 19:25:49.193843 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:49.193706    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-dhf4k"]
Oct 30 19:25:49.196515 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:25:49.196487    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8adead5a_6b26_4f4f_8d0e_c0a38120ca3b.slice/crio-0de013fc5200b8e41c1abe178202b8e1fb90623713e804c5488fe8d3bfcdecb6 WatchSource:0}: Error finding container 0de013fc5200b8e41c1abe178202b8e1fb90623713e804c5488fe8d3bfcdecb6: Status 404 returned error can't find the container with id 0de013fc5200b8e41c1abe178202b8e1fb90623713e804c5488fe8d3bfcdecb6
Oct 30 19:25:49.394378 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:49.394349    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5d58dddd7d-4txzv"
Oct 30 19:25:49.438018 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:49.437980    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-6sdjf" event={"ID":"c523ec8f-2430-498c-9a05-f7d5bf0a0a87","Type":"ContainerStarted","Data":"9be49145c3fa754fe6288ff766eeb46e32ca9c594cdbe8df8b79705e455b2cf8"}
Oct 30 19:25:49.439092 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:49.439072    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-dhf4k" event={"ID":"8adead5a-6b26-4f4f-8d0e-c0a38120ca3b","Type":"ContainerStarted","Data":"0de013fc5200b8e41c1abe178202b8e1fb90623713e804c5488fe8d3bfcdecb6"}
Oct 30 19:25:49.544616 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:49.544518    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-6sdjf" podStartSLOduration=2.006893599 podStartE2EDuration="4.544502503s" totalImagesPullingTime="2.537608904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:25:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:25:49.542710186 +0000 UTC m=+261.385642127" watchObservedRunningTime="2026-10-30 19:25:49.544502503 +0000 UTC m=+261.387434442"
Oct 30 19:25:52.455906 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:52.455854    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-dhf4k" event={"ID":"8adead5a-6b26-4f4f-8d0e-c0a38120ca3b","Type":"ContainerStarted","Data":"4ecf8dfc58b7a4e579336ba805a0b92580cf4dc525b0d80116def06eed958054"}
Oct 30 19:25:52.501873 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:52.501807    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-dhf4k" podStartSLOduration=1.695778861 podStartE2EDuration="4.501790367s" totalImagesPullingTime="2.806011506s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:25:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:25:52.501100884 +0000 UTC m=+264.344032822" watchObservedRunningTime="2026-10-30 19:25:52.501790367 +0000 UTC m=+264.344722305"
Oct 30 19:25:57.285693 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:57.285658    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-597b95cb84-q4928"]
Oct 30 19:25:57.289101 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:57.289085    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-q4928"
Oct 30 19:25:57.311716 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:57.311691    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-g2jnb\""
Oct 30 19:25:57.326431 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:57.326402    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-q4928"]
Oct 30 19:25:57.448702 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:57.448669    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45l6p\" (UniqueName: \"kubernetes.io/projected/6beea97b-24c5-493d-a863-49390020ce0f-kube-api-access-45l6p\") pod \"cert-manager-597b95cb84-q4928\" (UID: \"6beea97b-24c5-493d-a863-49390020ce0f\") " pod="cert-manager/cert-manager-597b95cb84-q4928"
Oct 30 19:25:57.448702 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:57.448708    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6beea97b-24c5-493d-a863-49390020ce0f-bound-sa-token\") pod \"cert-manager-597b95cb84-q4928\" (UID: \"6beea97b-24c5-493d-a863-49390020ce0f\") " pod="cert-manager/cert-manager-597b95cb84-q4928"
Oct 30 19:25:57.550313 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:57.550233    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45l6p\" (UniqueName: \"kubernetes.io/projected/6beea97b-24c5-493d-a863-49390020ce0f-kube-api-access-45l6p\") pod \"cert-manager-597b95cb84-q4928\" (UID: \"6beea97b-24c5-493d-a863-49390020ce0f\") " pod="cert-manager/cert-manager-597b95cb84-q4928"
Oct 30 19:25:57.550313 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:57.550292    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6beea97b-24c5-493d-a863-49390020ce0f-bound-sa-token\") pod \"cert-manager-597b95cb84-q4928\" (UID: \"6beea97b-24c5-493d-a863-49390020ce0f\") " pod="cert-manager/cert-manager-597b95cb84-q4928"
Oct 30 19:25:57.573437 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:57.573401    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6beea97b-24c5-493d-a863-49390020ce0f-bound-sa-token\") pod \"cert-manager-597b95cb84-q4928\" (UID: \"6beea97b-24c5-493d-a863-49390020ce0f\") " pod="cert-manager/cert-manager-597b95cb84-q4928"
Oct 30 19:25:57.575099 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:57.575078    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45l6p\" (UniqueName: \"kubernetes.io/projected/6beea97b-24c5-493d-a863-49390020ce0f-kube-api-access-45l6p\") pod \"cert-manager-597b95cb84-q4928\" (UID: \"6beea97b-24c5-493d-a863-49390020ce0f\") " pod="cert-manager/cert-manager-597b95cb84-q4928"
Oct 30 19:25:57.600944 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:57.600924    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-q4928"
Oct 30 19:25:57.909803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:57.909780    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-q4928"]
Oct 30 19:25:57.912505 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:25:57.912467    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6beea97b_24c5_493d_a863_49390020ce0f.slice/crio-1af96faae6e8adc01ddacc883192c27104044bfcce7ad186243728cf90dce18b WatchSource:0}: Error finding container 1af96faae6e8adc01ddacc883192c27104044bfcce7ad186243728cf90dce18b: Status 404 returned error can't find the container with id 1af96faae6e8adc01ddacc883192c27104044bfcce7ad186243728cf90dce18b
Oct 30 19:25:58.482042 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:58.482012    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-q4928" event={"ID":"6beea97b-24c5-493d-a863-49390020ce0f","Type":"ContainerStarted","Data":"91683a118c4341f32498815592d250207ceb73d1d426dcb90fe37244e091b53f"}
Oct 30 19:25:58.482042 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:58.482051    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-q4928" event={"ID":"6beea97b-24c5-493d-a863-49390020ce0f","Type":"ContainerStarted","Data":"1af96faae6e8adc01ddacc883192c27104044bfcce7ad186243728cf90dce18b"}
Oct 30 19:25:58.525355 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:25:58.525296    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-597b95cb84-q4928" podStartSLOduration=1.5252818910000001 podStartE2EDuration="1.525281891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:25:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:25:58.523995084 +0000 UTC m=+270.366927024" watchObservedRunningTime="2026-10-30 19:25:58.525281891 +0000 UTC m=+270.368213829"
Oct 30 19:26:08.720481 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:08.720408    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-58487bf9b4-h7x8l"]
Oct 30 19:26:08.723597 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:08.723581    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-58487bf9b4-h7x8l"
Oct 30 19:26:08.743846 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:08.743809    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Oct 30 19:26:08.756084 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:08.756052    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 19:26:08.756227 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:08.756158    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 19:26:08.756227 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:08.756175    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-9h9vk\""
Oct 30 19:26:08.756408 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:08.756229    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Oct 30 19:26:08.757901 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:08.757849    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/872f33d5-d35d-4bbf-b063-748f8dee26af-tls-certs\") pod \"kube-auth-proxy-58487bf9b4-h7x8l\" (UID: \"872f33d5-d35d-4bbf-b063-748f8dee26af\") " pod="openshift-ingress/kube-auth-proxy-58487bf9b4-h7x8l"
Oct 30 19:26:08.758022 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:08.758005    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/872f33d5-d35d-4bbf-b063-748f8dee26af-tmp\") pod \"kube-auth-proxy-58487bf9b4-h7x8l\" (UID: \"872f33d5-d35d-4bbf-b063-748f8dee26af\") " pod="openshift-ingress/kube-auth-proxy-58487bf9b4-h7x8l"
Oct 30 19:26:08.758077 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:08.758056    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kk4w\" (UniqueName: \"kubernetes.io/projected/872f33d5-d35d-4bbf-b063-748f8dee26af-kube-api-access-2kk4w\") pod \"kube-auth-proxy-58487bf9b4-h7x8l\" (UID: \"872f33d5-d35d-4bbf-b063-748f8dee26af\") " pod="openshift-ingress/kube-auth-proxy-58487bf9b4-h7x8l"
Oct 30 19:26:08.759340 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:08.759305    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-58487bf9b4-h7x8l"]
Oct 30 19:26:08.859549 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:08.859511    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/872f33d5-d35d-4bbf-b063-748f8dee26af-tls-certs\") pod \"kube-auth-proxy-58487bf9b4-h7x8l\" (UID: \"872f33d5-d35d-4bbf-b063-748f8dee26af\") " pod="openshift-ingress/kube-auth-proxy-58487bf9b4-h7x8l"
Oct 30 19:26:08.859736 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:08.859601    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/872f33d5-d35d-4bbf-b063-748f8dee26af-tmp\") pod \"kube-auth-proxy-58487bf9b4-h7x8l\" (UID: \"872f33d5-d35d-4bbf-b063-748f8dee26af\") " pod="openshift-ingress/kube-auth-proxy-58487bf9b4-h7x8l"
Oct 30 19:26:08.859736 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:08.859639    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kk4w\" (UniqueName: \"kubernetes.io/projected/872f33d5-d35d-4bbf-b063-748f8dee26af-kube-api-access-2kk4w\") pod \"kube-auth-proxy-58487bf9b4-h7x8l\" (UID: \"872f33d5-d35d-4bbf-b063-748f8dee26af\") " pod="openshift-ingress/kube-auth-proxy-58487bf9b4-h7x8l"
Oct 30 19:26:08.862172 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:08.862143    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/872f33d5-d35d-4bbf-b063-748f8dee26af-tmp\") pod \"kube-auth-proxy-58487bf9b4-h7x8l\" (UID: \"872f33d5-d35d-4bbf-b063-748f8dee26af\") " pod="openshift-ingress/kube-auth-proxy-58487bf9b4-h7x8l"
Oct 30 19:26:08.862320 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:08.862304    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/872f33d5-d35d-4bbf-b063-748f8dee26af-tls-certs\") pod \"kube-auth-proxy-58487bf9b4-h7x8l\" (UID: \"872f33d5-d35d-4bbf-b063-748f8dee26af\") " pod="openshift-ingress/kube-auth-proxy-58487bf9b4-h7x8l"
Oct 30 19:26:08.888146 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:08.888121    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kk4w\" (UniqueName: \"kubernetes.io/projected/872f33d5-d35d-4bbf-b063-748f8dee26af-kube-api-access-2kk4w\") pod \"kube-auth-proxy-58487bf9b4-h7x8l\" (UID: \"872f33d5-d35d-4bbf-b063-748f8dee26af\") " pod="openshift-ingress/kube-auth-proxy-58487bf9b4-h7x8l"
Oct 30 19:26:09.036229 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:09.036155    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-58487bf9b4-h7x8l"
Oct 30 19:26:09.348730 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:09.348707    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-58487bf9b4-h7x8l"]
Oct 30 19:26:09.351146 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:26:09.351119    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod872f33d5_d35d_4bbf_b063_748f8dee26af.slice/crio-a5586c19b59746523bacee328317f8b4eafbf08e8f45f8ddbb8c3e21434d4381 WatchSource:0}: Error finding container a5586c19b59746523bacee328317f8b4eafbf08e8f45f8ddbb8c3e21434d4381: Status 404 returned error can't find the container with id a5586c19b59746523bacee328317f8b4eafbf08e8f45f8ddbb8c3e21434d4381
Oct 30 19:26:09.527941 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:09.527901    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-58487bf9b4-h7x8l" event={"ID":"872f33d5-d35d-4bbf-b063-748f8dee26af","Type":"ContainerStarted","Data":"a5586c19b59746523bacee328317f8b4eafbf08e8f45f8ddbb8c3e21434d4381"}
Oct 30 19:26:09.736280 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:09.736245    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-5sfvj"]
Oct 30 19:26:09.740709 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:09.740688    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-5sfvj"
Oct 30 19:26:09.767386 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:09.767355    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9r9d\" (UniqueName: \"kubernetes.io/projected/6e29c707-d325-4076-b3e7-b6efcae71584-kube-api-access-s9r9d\") pod \"limitador-limitador-74dfcfb894-5sfvj\" (UID: \"6e29c707-d325-4076-b3e7-b6efcae71584\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-5sfvj"
Oct 30 19:26:09.767566 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:09.767550    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6e29c707-d325-4076-b3e7-b6efcae71584-config-file\") pod \"limitador-limitador-74dfcfb894-5sfvj\" (UID: \"6e29c707-d325-4076-b3e7-b6efcae71584\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-5sfvj"
Oct 30 19:26:09.768562 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:09.768536    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-5sfvj"]
Oct 30 19:26:09.781198 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:09.780977    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 30 19:26:09.781198 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:09.781021    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Oct 30 19:26:09.781198 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:09.781042    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-nrqr9\""
Oct 30 19:26:09.781198 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:09.781097    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 30 19:26:09.868138 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:09.868093    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9r9d\" (UniqueName: \"kubernetes.io/projected/6e29c707-d325-4076-b3e7-b6efcae71584-kube-api-access-s9r9d\") pod \"limitador-limitador-74dfcfb894-5sfvj\" (UID: \"6e29c707-d325-4076-b3e7-b6efcae71584\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-5sfvj"
Oct 30 19:26:09.868138 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:09.868139    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6e29c707-d325-4076-b3e7-b6efcae71584-config-file\") pod \"limitador-limitador-74dfcfb894-5sfvj\" (UID: \"6e29c707-d325-4076-b3e7-b6efcae71584\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-5sfvj"
Oct 30 19:26:09.868926 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:09.868901    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6e29c707-d325-4076-b3e7-b6efcae71584-config-file\") pod \"limitador-limitador-74dfcfb894-5sfvj\" (UID: \"6e29c707-d325-4076-b3e7-b6efcae71584\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-5sfvj"
Oct 30 19:26:09.872607 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:09.872579    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-5sfvj"]
Oct 30 19:26:09.891964 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:09.891930    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9r9d\" (UniqueName: \"kubernetes.io/projected/6e29c707-d325-4076-b3e7-b6efcae71584-kube-api-access-s9r9d\") pod \"limitador-limitador-74dfcfb894-5sfvj\" (UID: \"6e29c707-d325-4076-b3e7-b6efcae71584\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-5sfvj"
Oct 30 19:26:10.054008 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:10.053919    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-5sfvj"
Oct 30 19:26:10.377277 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:10.377251    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-5sfvj"]
Oct 30 19:26:10.391441 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:26:10.391409    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e29c707_d325_4076_b3e7_b6efcae71584.slice/crio-5b60e959dde7ce0e52064fe123eb93f60dd04ec08f7ef715659de86dd96184bc WatchSource:0}: Error finding container 5b60e959dde7ce0e52064fe123eb93f60dd04ec08f7ef715659de86dd96184bc: Status 404 returned error can't find the container with id 5b60e959dde7ce0e52064fe123eb93f60dd04ec08f7ef715659de86dd96184bc
Oct 30 19:26:10.535330 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:10.535287    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-5sfvj" event={"ID":"6e29c707-d325-4076-b3e7-b6efcae71584","Type":"ContainerStarted","Data":"5b60e959dde7ce0e52064fe123eb93f60dd04ec08f7ef715659de86dd96184bc"}
Oct 30 19:26:10.636898 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:10.636818    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57f7f8659f-jcb6b"]
Oct 30 19:26:10.641679 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:10.641657    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-jcb6b"
Oct 30 19:26:10.661543 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:10.661515    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-jcb6b"]
Oct 30 19:26:10.683040 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:10.683013    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlqz7\" (UniqueName: \"kubernetes.io/projected/8e9a6670-c511-4d80-831a-4cc478321610-kube-api-access-dlqz7\") pod \"authorino-57f7f8659f-jcb6b\" (UID: \"8e9a6670-c511-4d80-831a-4cc478321610\") " pod="kuadrant-system/authorino-57f7f8659f-jcb6b"
Oct 30 19:26:10.690735 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:10.690712    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-dvjb7\""
Oct 30 19:26:10.783666 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:10.783620    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlqz7\" (UniqueName: \"kubernetes.io/projected/8e9a6670-c511-4d80-831a-4cc478321610-kube-api-access-dlqz7\") pod \"authorino-57f7f8659f-jcb6b\" (UID: \"8e9a6670-c511-4d80-831a-4cc478321610\") " pod="kuadrant-system/authorino-57f7f8659f-jcb6b"
Oct 30 19:26:10.805140 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:10.805113    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlqz7\" (UniqueName: \"kubernetes.io/projected/8e9a6670-c511-4d80-831a-4cc478321610-kube-api-access-dlqz7\") pod \"authorino-57f7f8659f-jcb6b\" (UID: \"8e9a6670-c511-4d80-831a-4cc478321610\") " pod="kuadrant-system/authorino-57f7f8659f-jcb6b"
Oct 30 19:26:10.957469 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:10.957383    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-jcb6b"
Oct 30 19:26:11.202837 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:11.202801    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:26:11.208733 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:11.208673    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-28zhj"
Oct 30 19:26:11.231014 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:11.230946    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:26:11.286898 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:11.286852    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:26:11.288463 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:11.288437    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzmxg\" (UniqueName: \"kubernetes.io/projected/3e6bddcf-4a27-42f8-bc2a-f68e038ab415-kube-api-access-dzmxg\") pod \"limitador-limitador-7c9b87d574-28zhj\" (UID: \"3e6bddcf-4a27-42f8-bc2a-f68e038ab415\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-28zhj"
Oct 30 19:26:11.288600 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:11.288484    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3e6bddcf-4a27-42f8-bc2a-f68e038ab415-config-file\") pod \"limitador-limitador-7c9b87d574-28zhj\" (UID: \"3e6bddcf-4a27-42f8-bc2a-f68e038ab415\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-28zhj"
Oct 30 19:26:11.297387 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:11.297357    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-jcb6b"]
Oct 30 19:26:11.301651 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:26:11.301625    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8e9a6670_c511_4d80_831a_4cc478321610.slice/crio-e3fa15d04342fdc0c1c86c61f6bb01145c6af462a947b6ab243897d601be9a44 WatchSource:0}: Error finding container e3fa15d04342fdc0c1c86c61f6bb01145c6af462a947b6ab243897d601be9a44: Status 404 returned error can't find the container with id e3fa15d04342fdc0c1c86c61f6bb01145c6af462a947b6ab243897d601be9a44
Oct 30 19:26:11.389678 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:11.389450    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzmxg\" (UniqueName: \"kubernetes.io/projected/3e6bddcf-4a27-42f8-bc2a-f68e038ab415-kube-api-access-dzmxg\") pod \"limitador-limitador-7c9b87d574-28zhj\" (UID: \"3e6bddcf-4a27-42f8-bc2a-f68e038ab415\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-28zhj"
Oct 30 19:26:11.389678 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:11.389548    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3e6bddcf-4a27-42f8-bc2a-f68e038ab415-config-file\") pod \"limitador-limitador-7c9b87d574-28zhj\" (UID: \"3e6bddcf-4a27-42f8-bc2a-f68e038ab415\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-28zhj"
Oct 30 19:26:11.390418 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:11.390390    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3e6bddcf-4a27-42f8-bc2a-f68e038ab415-config-file\") pod \"limitador-limitador-7c9b87d574-28zhj\" (UID: \"3e6bddcf-4a27-42f8-bc2a-f68e038ab415\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-28zhj"
Oct 30 19:26:11.412892 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:11.412053    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzmxg\" (UniqueName: \"kubernetes.io/projected/3e6bddcf-4a27-42f8-bc2a-f68e038ab415-kube-api-access-dzmxg\") pod \"limitador-limitador-7c9b87d574-28zhj\" (UID: \"3e6bddcf-4a27-42f8-bc2a-f68e038ab415\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-28zhj"
Oct 30 19:26:11.527691 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:11.527596    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-28zhj"
Oct 30 19:26:11.544408 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:11.544373    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-jcb6b" event={"ID":"8e9a6670-c511-4d80-831a-4cc478321610","Type":"ContainerStarted","Data":"e3fa15d04342fdc0c1c86c61f6bb01145c6af462a947b6ab243897d601be9a44"}
Oct 30 19:26:11.834446 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:11.834314    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:26:11.837288 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:26:11.837249    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e6bddcf_4a27_42f8_bc2a_f68e038ab415.slice/crio-b3bb5cbadcd904606fe053c1563ea57c7d34407ff8602cfc4dbc070f9499d3fc WatchSource:0}: Error finding container b3bb5cbadcd904606fe053c1563ea57c7d34407ff8602cfc4dbc070f9499d3fc: Status 404 returned error can't find the container with id b3bb5cbadcd904606fe053c1563ea57c7d34407ff8602cfc4dbc070f9499d3fc
Oct 30 19:26:12.556124 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:12.556085    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-58487bf9b4-h7x8l" event={"ID":"872f33d5-d35d-4bbf-b063-748f8dee26af","Type":"ContainerStarted","Data":"479888c85aa9dcfe307be3c0e4553c51c308d0ef27f6d25a2d780fe40cc70da1"}
Oct 30 19:26:12.559496 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:12.559462    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-28zhj" event={"ID":"3e6bddcf-4a27-42f8-bc2a-f68e038ab415","Type":"ContainerStarted","Data":"b3bb5cbadcd904606fe053c1563ea57c7d34407ff8602cfc4dbc070f9499d3fc"}
Oct 30 19:26:12.607022 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:12.605808    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-58487bf9b4-h7x8l" podStartSLOduration=2.071213596 podStartE2EDuration="4.605790264s" totalImagesPullingTime="2.534576668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:26:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:26:12.604944803 +0000 UTC m=+284.447876743" watchObservedRunningTime="2026-10-30 19:26:12.605790264 +0000 UTC m=+284.448722205"
Oct 30 19:26:13.552111 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:13.552078    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-jcb6b"]
Oct 30 19:26:13.566193 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:13.566159    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-28zhj" event={"ID":"3e6bddcf-4a27-42f8-bc2a-f68e038ab415","Type":"ContainerStarted","Data":"577f5154744e982faeea3c07aa2cde7bbed3e9cb6bff8e5caf4abe316c283570"}
Oct 30 19:26:13.566385 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:13.566277    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-28zhj"
Oct 30 19:26:13.568189 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:13.568163    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-5sfvj" event={"ID":"6e29c707-d325-4076-b3e7-b6efcae71584","Type":"ContainerStarted","Data":"425f9102774f20ccaa63d7cfb28a1d1bf3af4eb84ac67bdcf3d4a5c60c7fa754"}
Oct 30 19:26:13.609183 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:13.609125    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7c9b87d574-28zhj" podStartSLOduration=1.03985075 podStartE2EDuration="2.60910563s" totalImagesPullingTime="1.56925488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:26:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:26:13.608271881 +0000 UTC m=+285.451203822" watchObservedRunningTime="2026-10-30 19:26:13.60910563 +0000 UTC m=+285.452037583"
Oct 30 19:26:13.650245 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:13.650147    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-5sfvj" podStartSLOduration=1.644495373 podStartE2EDuration="4.650127713s" totalImagesPullingTime="3.00563234s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:26:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:26:13.648747097 +0000 UTC m=+285.491679037" watchObservedRunningTime="2026-10-30 19:26:13.650127713 +0000 UTC m=+285.493059654"
Oct 30 19:26:14.574877 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:14.574770    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-jcb6b" event={"ID":"8e9a6670-c511-4d80-831a-4cc478321610","Type":"ContainerStarted","Data":"1f8cf388cb1e7d3470399d050fddef47563cca391e39bb21896a6230019da644"}
Oct 30 19:26:14.574877 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:14.574784    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-57f7f8659f-jcb6b" podUID="8e9a6670-c511-4d80-831a-4cc478321610" containerName="authorino" containerID="cri-o://1f8cf388cb1e7d3470399d050fddef47563cca391e39bb21896a6230019da644" gracePeriod=30
Oct 30 19:26:14.575561 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:14.575546    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-5sfvj"
Oct 30 19:26:14.616019 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:14.615973    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-57f7f8659f-jcb6b" podStartSLOduration=1.632681429 podStartE2EDuration="4.615957362s" totalImagesPullingTime="2.983275933s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:26:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:26:14.615206967 +0000 UTC m=+286.458138906" watchObservedRunningTime="2026-10-30 19:26:14.615957362 +0000 UTC m=+286.458889300"
Oct 30 19:26:14.826028 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:14.825969    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-jcb6b"
Oct 30 19:26:14.932932 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:14.932901    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8e9a6670-c511-4d80-831a-4cc478321610-kube-api-access-dlqz7\" (UniqueName: \"kubernetes.io/projected/8e9a6670-c511-4d80-831a-4cc478321610-kube-api-access-dlqz7\") pod \"8e9a6670-c511-4d80-831a-4cc478321610\" (UID: \"8e9a6670-c511-4d80-831a-4cc478321610\") "
Oct 30 19:26:14.935369 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:14.935338    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e9a6670-c511-4d80-831a-4cc478321610-kube-api-access-dlqz7" pod "8e9a6670-c511-4d80-831a-4cc478321610" (UID: "8e9a6670-c511-4d80-831a-4cc478321610"). InnerVolumeSpecName "kube-api-access-dlqz7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:26:15.034137 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:15.034100    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dlqz7\" (UniqueName: \"kubernetes.io/projected/8e9a6670-c511-4d80-831a-4cc478321610-kube-api-access-dlqz7\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:26:15.580956 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:15.580924    5907 generic.go:356] "Generic (PLEG): container finished" podID="8e9a6670-c511-4d80-831a-4cc478321610" containerID="1f8cf388cb1e7d3470399d050fddef47563cca391e39bb21896a6230019da644" exitCode=0
Oct 30 19:26:15.581384 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:15.580984    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-jcb6b"
Oct 30 19:26:15.581384 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:15.581033    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-jcb6b" event={"ID":"8e9a6670-c511-4d80-831a-4cc478321610","Type":"ContainerDied","Data":"1f8cf388cb1e7d3470399d050fddef47563cca391e39bb21896a6230019da644"}
Oct 30 19:26:15.581384 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:15.581078    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-jcb6b" event={"ID":"8e9a6670-c511-4d80-831a-4cc478321610","Type":"ContainerDied","Data":"e3fa15d04342fdc0c1c86c61f6bb01145c6af462a947b6ab243897d601be9a44"}
Oct 30 19:26:15.581384 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:15.581102    5907 scope.go:122] "RemoveContainer" containerID="1f8cf388cb1e7d3470399d050fddef47563cca391e39bb21896a6230019da644"
Oct 30 19:26:15.589581 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:15.589566    5907 scope.go:122] "RemoveContainer" containerID="1f8cf388cb1e7d3470399d050fddef47563cca391e39bb21896a6230019da644"
Oct 30 19:26:15.589887 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:26:15.589845    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f8cf388cb1e7d3470399d050fddef47563cca391e39bb21896a6230019da644\": container with ID starting with 1f8cf388cb1e7d3470399d050fddef47563cca391e39bb21896a6230019da644 not found: ID does not exist" containerID="1f8cf388cb1e7d3470399d050fddef47563cca391e39bb21896a6230019da644"
Oct 30 19:26:15.589948 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:15.589894    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f8cf388cb1e7d3470399d050fddef47563cca391e39bb21896a6230019da644"} err="failed to get container status \"1f8cf388cb1e7d3470399d050fddef47563cca391e39bb21896a6230019da644\": rpc error: code = NotFound desc = could not find container \"1f8cf388cb1e7d3470399d050fddef47563cca391e39bb21896a6230019da644\": container with ID starting with 1f8cf388cb1e7d3470399d050fddef47563cca391e39bb21896a6230019da644 not found: ID does not exist"
Oct 30 19:26:15.640115 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:15.640085    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-jcb6b"]
Oct 30 19:26:15.653672 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:15.653643    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-jcb6b"]
Oct 30 19:26:16.285625 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:16.285590    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8e9a6670-c511-4d80-831a-4cc478321610" path="/var/lib/kubelet/pods/8e9a6670-c511-4d80-831a-4cc478321610/volumes"
Oct 30 19:26:17.500517 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:26:17.500479    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:26:24.576001 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:24.575970    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-28zhj"
Oct 30 19:26:24.641092 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:24.641059    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-5sfvj"]
Oct 30 19:26:24.641259 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:24.641234    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-5sfvj" podUID="6e29c707-d325-4076-b3e7-b6efcae71584" containerName="limitador" containerID="cri-o://425f9102774f20ccaa63d7cfb28a1d1bf3af4eb84ac67bdcf3d4a5c60c7fa754" gracePeriod=30
Oct 30 19:26:24.641982 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:24.641920    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-5sfvj"
Oct 30 19:26:25.572789 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:25.572758    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-5sfvj"
Oct 30 19:26:25.627124 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:25.627082    5907 generic.go:356] "Generic (PLEG): container finished" podID="6e29c707-d325-4076-b3e7-b6efcae71584" containerID="425f9102774f20ccaa63d7cfb28a1d1bf3af4eb84ac67bdcf3d4a5c60c7fa754" exitCode=0
Oct 30 19:26:25.627124 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:25.627125    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-5sfvj"
Oct 30 19:26:25.627566 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:25.627164    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-5sfvj" event={"ID":"6e29c707-d325-4076-b3e7-b6efcae71584","Type":"ContainerDied","Data":"425f9102774f20ccaa63d7cfb28a1d1bf3af4eb84ac67bdcf3d4a5c60c7fa754"}
Oct 30 19:26:25.627566 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:25.627275    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-5sfvj" event={"ID":"6e29c707-d325-4076-b3e7-b6efcae71584","Type":"ContainerDied","Data":"5b60e959dde7ce0e52064fe123eb93f60dd04ec08f7ef715659de86dd96184bc"}
Oct 30 19:26:25.627566 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:25.627308    5907 scope.go:122] "RemoveContainer" containerID="425f9102774f20ccaa63d7cfb28a1d1bf3af4eb84ac67bdcf3d4a5c60c7fa754"
Oct 30 19:26:25.636486 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:25.636467    5907 scope.go:122] "RemoveContainer" containerID="425f9102774f20ccaa63d7cfb28a1d1bf3af4eb84ac67bdcf3d4a5c60c7fa754"
Oct 30 19:26:25.636765 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:26:25.636746    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"425f9102774f20ccaa63d7cfb28a1d1bf3af4eb84ac67bdcf3d4a5c60c7fa754\": container with ID starting with 425f9102774f20ccaa63d7cfb28a1d1bf3af4eb84ac67bdcf3d4a5c60c7fa754 not found: ID does not exist" containerID="425f9102774f20ccaa63d7cfb28a1d1bf3af4eb84ac67bdcf3d4a5c60c7fa754"
Oct 30 19:26:25.636843 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:25.636773    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"425f9102774f20ccaa63d7cfb28a1d1bf3af4eb84ac67bdcf3d4a5c60c7fa754"} err="failed to get container status \"425f9102774f20ccaa63d7cfb28a1d1bf3af4eb84ac67bdcf3d4a5c60c7fa754\": rpc error: code = NotFound desc = could not find container \"425f9102774f20ccaa63d7cfb28a1d1bf3af4eb84ac67bdcf3d4a5c60c7fa754\": container with ID starting with 425f9102774f20ccaa63d7cfb28a1d1bf3af4eb84ac67bdcf3d4a5c60c7fa754 not found: ID does not exist"
Oct 30 19:26:25.734854 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:25.734772    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6e29c707-d325-4076-b3e7-b6efcae71584-config-file\" (UniqueName: \"kubernetes.io/configmap/6e29c707-d325-4076-b3e7-b6efcae71584-config-file\") pod \"6e29c707-d325-4076-b3e7-b6efcae71584\" (UID: \"6e29c707-d325-4076-b3e7-b6efcae71584\") "
Oct 30 19:26:25.734854 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:25.734828    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6e29c707-d325-4076-b3e7-b6efcae71584-kube-api-access-s9r9d\" (UniqueName: \"kubernetes.io/projected/6e29c707-d325-4076-b3e7-b6efcae71584-kube-api-access-s9r9d\") pod \"6e29c707-d325-4076-b3e7-b6efcae71584\" (UID: \"6e29c707-d325-4076-b3e7-b6efcae71584\") "
Oct 30 19:26:25.735234 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:25.735209    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e29c707-d325-4076-b3e7-b6efcae71584-config-file" pod "6e29c707-d325-4076-b3e7-b6efcae71584" (UID: "6e29c707-d325-4076-b3e7-b6efcae71584"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:26:25.737045 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:25.737027    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e29c707-d325-4076-b3e7-b6efcae71584-kube-api-access-s9r9d" pod "6e29c707-d325-4076-b3e7-b6efcae71584" (UID: "6e29c707-d325-4076-b3e7-b6efcae71584"). InnerVolumeSpecName "kube-api-access-s9r9d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:26:25.835670 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:25.835637    5907 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6e29c707-d325-4076-b3e7-b6efcae71584-config-file\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:26:25.835670 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:25.835667    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9r9d\" (UniqueName: \"kubernetes.io/projected/6e29c707-d325-4076-b3e7-b6efcae71584-kube-api-access-s9r9d\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:26:25.976398 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:25.976364    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-5sfvj"]
Oct 30 19:26:25.991042 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:25.990976    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-5sfvj"]
Oct 30 19:26:26.287279 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:26.287203    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6e29c707-d325-4076-b3e7-b6efcae71584" path="/var/lib/kubelet/pods/6e29c707-d325-4076-b3e7-b6efcae71584/volumes"
Oct 30 19:26:31.214201 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:31.214164    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-wjf8m"]
Oct 30 19:26:31.214589 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:31.214539    5907 cpu_manager.go:414] "containerMap: removing container" podUID="8e9a6670-c511-4d80-831a-4cc478321610" containerName="authorino"
Oct 30 19:26:31.214589 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:31.214550    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8e9a6670-c511-4d80-831a-4cc478321610" containerName="authorino"
Oct 30 19:26:31.214589 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:31.214565    5907 cpu_manager.go:414] "containerMap: removing container" podUID="6e29c707-d325-4076-b3e7-b6efcae71584" containerName="limitador"
Oct 30 19:26:31.214589 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:31.214570    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6e29c707-d325-4076-b3e7-b6efcae71584" containerName="limitador"
Oct 30 19:26:31.214762 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:31.214638    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="6e29c707-d325-4076-b3e7-b6efcae71584" containerName="limitador"
Oct 30 19:26:31.214762 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:31.214651    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="8e9a6670-c511-4d80-831a-4cc478321610" containerName="authorino"
Oct 30 19:26:31.217746 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:31.217724    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wjf8m"
Oct 30 19:26:31.238212 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:31.238182    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 19:26:31.238340 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:31.238235    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-wjf8m"]
Oct 30 19:26:31.250757 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:31.250736    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 19:26:31.296715 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:31.296686    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Oct 30 19:26:31.296715 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:31.296716    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-58tc2\""
Oct 30 19:26:31.386382 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:31.386346    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5c355944-4ac3-4538-9430-72cd55cb9d1e-data\") pod \"postgres-7cdbf6dc44-wjf8m\" (UID: \"5c355944-4ac3-4538-9430-72cd55cb9d1e\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wjf8m"
Oct 30 19:26:31.386558 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:31.386426    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc79w\" (UniqueName: \"kubernetes.io/projected/5c355944-4ac3-4538-9430-72cd55cb9d1e-kube-api-access-zc79w\") pod \"postgres-7cdbf6dc44-wjf8m\" (UID: \"5c355944-4ac3-4538-9430-72cd55cb9d1e\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wjf8m"
Oct 30 19:26:31.487360 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:31.487277    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc79w\" (UniqueName: \"kubernetes.io/projected/5c355944-4ac3-4538-9430-72cd55cb9d1e-kube-api-access-zc79w\") pod \"postgres-7cdbf6dc44-wjf8m\" (UID: \"5c355944-4ac3-4538-9430-72cd55cb9d1e\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wjf8m"
Oct 30 19:26:31.487360 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:31.487334    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5c355944-4ac3-4538-9430-72cd55cb9d1e-data\") pod \"postgres-7cdbf6dc44-wjf8m\" (UID: \"5c355944-4ac3-4538-9430-72cd55cb9d1e\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wjf8m"
Oct 30 19:26:31.487664 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:31.487649    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5c355944-4ac3-4538-9430-72cd55cb9d1e-data\") pod \"postgres-7cdbf6dc44-wjf8m\" (UID: \"5c355944-4ac3-4538-9430-72cd55cb9d1e\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wjf8m"
Oct 30 19:26:31.509180 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:31.509154    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc79w\" (UniqueName: \"kubernetes.io/projected/5c355944-4ac3-4538-9430-72cd55cb9d1e-kube-api-access-zc79w\") pod \"postgres-7cdbf6dc44-wjf8m\" (UID: \"5c355944-4ac3-4538-9430-72cd55cb9d1e\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wjf8m"
Oct 30 19:26:31.529634 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:31.529610    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wjf8m"
Oct 30 19:26:31.825392 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:31.825369    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-wjf8m"]
Oct 30 19:26:31.827842 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:26:31.827800    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c355944_4ac3_4538_9430_72cd55cb9d1e.slice/crio-25711f0766e42023ab83910cb145a9c4ed6add30571fab84e82da38faab6edee WatchSource:0}: Error finding container 25711f0766e42023ab83910cb145a9c4ed6add30571fab84e82da38faab6edee: Status 404 returned error can't find the container with id 25711f0766e42023ab83910cb145a9c4ed6add30571fab84e82da38faab6edee
Oct 30 19:26:32.663226 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:32.663184    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wjf8m" event={"ID":"5c355944-4ac3-4538-9430-72cd55cb9d1e","Type":"ContainerStarted","Data":"25711f0766e42023ab83910cb145a9c4ed6add30571fab84e82da38faab6edee"}
Oct 30 19:26:37.688507 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:37.688470    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wjf8m" event={"ID":"5c355944-4ac3-4538-9430-72cd55cb9d1e","Type":"ContainerStarted","Data":"ae0cb17030c09913f416d0dbb65db03f46d781006b744b78f271c9742747e62a"}
Oct 30 19:26:37.689035 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:37.688598    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wjf8m"
Oct 30 19:26:37.734286 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:37.734229    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wjf8m" podStartSLOduration=1.91331207 podStartE2EDuration="6.734215277s" totalImagesPullingTime="4.820903207s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:26:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:26:37.731922934 +0000 UTC m=+309.574854872" watchObservedRunningTime="2026-10-30 19:26:37.734215277 +0000 UTC m=+309.577147215"
Oct 30 19:26:43.723448 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:43.723419    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wjf8m"
Oct 30 19:26:47.499923 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:26:47.499879    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:26:50.744294 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:50.744266    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-579cf7c6d-5m84s"]
Oct 30 19:26:50.763605 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:50.763575    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-5m84s"
Oct 30 19:26:50.777152 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:50.777122    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-5m84s"]
Oct 30 19:26:50.788495 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:50.788457    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 19:26:50.788655 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:50.788522    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 30 19:26:50.793768 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:50.793744    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 19:26:50.793902 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:50.793787    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-58qcr\""
Oct 30 19:26:50.793902 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:50.793801    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 30 19:26:50.793902 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:50.793811    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 19:26:50.884028 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:50.883994    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxplz\" (UniqueName: \"kubernetes.io/projected/9baa9845-7af3-46b4-bdc3-368a9332dbc2-kube-api-access-vxplz\") pod \"maas-controller-579cf7c6d-5m84s\" (UID: \"9baa9845-7af3-46b4-bdc3-368a9332dbc2\") " pod="opendatahub/maas-controller-579cf7c6d-5m84s"
Oct 30 19:26:50.884193 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:50.884038    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9baa9845-7af3-46b4-bdc3-368a9332dbc2-metrics-certs\") pod \"maas-controller-579cf7c6d-5m84s\" (UID: \"9baa9845-7af3-46b4-bdc3-368a9332dbc2\") " pod="opendatahub/maas-controller-579cf7c6d-5m84s"
Oct 30 19:26:50.884193 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:50.884155    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9baa9845-7af3-46b4-bdc3-368a9332dbc2-webhook-cert\") pod \"maas-controller-579cf7c6d-5m84s\" (UID: \"9baa9845-7af3-46b4-bdc3-368a9332dbc2\") " pod="opendatahub/maas-controller-579cf7c6d-5m84s"
Oct 30 19:26:50.985515 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:50.985483    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxplz\" (UniqueName: \"kubernetes.io/projected/9baa9845-7af3-46b4-bdc3-368a9332dbc2-kube-api-access-vxplz\") pod \"maas-controller-579cf7c6d-5m84s\" (UID: \"9baa9845-7af3-46b4-bdc3-368a9332dbc2\") " pod="opendatahub/maas-controller-579cf7c6d-5m84s"
Oct 30 19:26:50.985717 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:50.985524    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9baa9845-7af3-46b4-bdc3-368a9332dbc2-metrics-certs\") pod \"maas-controller-579cf7c6d-5m84s\" (UID: \"9baa9845-7af3-46b4-bdc3-368a9332dbc2\") " pod="opendatahub/maas-controller-579cf7c6d-5m84s"
Oct 30 19:26:50.985717 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:50.985653    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9baa9845-7af3-46b4-bdc3-368a9332dbc2-webhook-cert\") pod \"maas-controller-579cf7c6d-5m84s\" (UID: \"9baa9845-7af3-46b4-bdc3-368a9332dbc2\") " pod="opendatahub/maas-controller-579cf7c6d-5m84s"
Oct 30 19:26:50.988030 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:50.988008    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9baa9845-7af3-46b4-bdc3-368a9332dbc2-metrics-certs\") pod \"maas-controller-579cf7c6d-5m84s\" (UID: \"9baa9845-7af3-46b4-bdc3-368a9332dbc2\") " pod="opendatahub/maas-controller-579cf7c6d-5m84s"
Oct 30 19:26:50.988157 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:50.988065    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9baa9845-7af3-46b4-bdc3-368a9332dbc2-webhook-cert\") pod \"maas-controller-579cf7c6d-5m84s\" (UID: \"9baa9845-7af3-46b4-bdc3-368a9332dbc2\") " pod="opendatahub/maas-controller-579cf7c6d-5m84s"
Oct 30 19:26:51.006279 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:51.006210    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxplz\" (UniqueName: \"kubernetes.io/projected/9baa9845-7af3-46b4-bdc3-368a9332dbc2-kube-api-access-vxplz\") pod \"maas-controller-579cf7c6d-5m84s\" (UID: \"9baa9845-7af3-46b4-bdc3-368a9332dbc2\") " pod="opendatahub/maas-controller-579cf7c6d-5m84s"
Oct 30 19:26:51.071599 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:51.071567    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6b9b54ffb7-7xdf5"]
Oct 30 19:26:51.074101 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:51.074074    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b9b54ffb7-7xdf5"
Oct 30 19:26:51.076283 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:51.076265    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-5m84s"
Oct 30 19:26:51.098462 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:51.098427    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b9b54ffb7-7xdf5"]
Oct 30 19:26:51.187161 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:51.187126    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfh7c\" (UniqueName: \"kubernetes.io/projected/6be54382-a07f-4e83-949e-3bf550c46599-kube-api-access-dfh7c\") pod \"maas-controller-6b9b54ffb7-7xdf5\" (UID: \"6be54382-a07f-4e83-949e-3bf550c46599\") " pod="opendatahub/maas-controller-6b9b54ffb7-7xdf5"
Oct 30 19:26:51.187311 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:51.187185    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6be54382-a07f-4e83-949e-3bf550c46599-metrics-certs\") pod \"maas-controller-6b9b54ffb7-7xdf5\" (UID: \"6be54382-a07f-4e83-949e-3bf550c46599\") " pod="opendatahub/maas-controller-6b9b54ffb7-7xdf5"
Oct 30 19:26:51.187357 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:51.187309    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6be54382-a07f-4e83-949e-3bf550c46599-webhook-cert\") pod \"maas-controller-6b9b54ffb7-7xdf5\" (UID: \"6be54382-a07f-4e83-949e-3bf550c46599\") " pod="opendatahub/maas-controller-6b9b54ffb7-7xdf5"
Oct 30 19:26:51.288150 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:51.288062    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfh7c\" (UniqueName: \"kubernetes.io/projected/6be54382-a07f-4e83-949e-3bf550c46599-kube-api-access-dfh7c\") pod \"maas-controller-6b9b54ffb7-7xdf5\" (UID: \"6be54382-a07f-4e83-949e-3bf550c46599\") " pod="opendatahub/maas-controller-6b9b54ffb7-7xdf5"
Oct 30 19:26:51.288150 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:51.288130    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6be54382-a07f-4e83-949e-3bf550c46599-metrics-certs\") pod \"maas-controller-6b9b54ffb7-7xdf5\" (UID: \"6be54382-a07f-4e83-949e-3bf550c46599\") " pod="opendatahub/maas-controller-6b9b54ffb7-7xdf5"
Oct 30 19:26:51.288384 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:51.288190    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6be54382-a07f-4e83-949e-3bf550c46599-webhook-cert\") pod \"maas-controller-6b9b54ffb7-7xdf5\" (UID: \"6be54382-a07f-4e83-949e-3bf550c46599\") " pod="opendatahub/maas-controller-6b9b54ffb7-7xdf5"
Oct 30 19:26:51.290906 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:51.290882    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6be54382-a07f-4e83-949e-3bf550c46599-webhook-cert\") pod \"maas-controller-6b9b54ffb7-7xdf5\" (UID: \"6be54382-a07f-4e83-949e-3bf550c46599\") " pod="opendatahub/maas-controller-6b9b54ffb7-7xdf5"
Oct 30 19:26:51.291021 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:51.290949    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6be54382-a07f-4e83-949e-3bf550c46599-metrics-certs\") pod \"maas-controller-6b9b54ffb7-7xdf5\" (UID: \"6be54382-a07f-4e83-949e-3bf550c46599\") " pod="opendatahub/maas-controller-6b9b54ffb7-7xdf5"
Oct 30 19:26:51.309247 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:51.309218    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfh7c\" (UniqueName: \"kubernetes.io/projected/6be54382-a07f-4e83-949e-3bf550c46599-kube-api-access-dfh7c\") pod \"maas-controller-6b9b54ffb7-7xdf5\" (UID: \"6be54382-a07f-4e83-949e-3bf550c46599\") " pod="opendatahub/maas-controller-6b9b54ffb7-7xdf5"
Oct 30 19:26:51.375114 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:51.375081    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-5m84s"]
Oct 30 19:26:51.377519 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:26:51.377492    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9baa9845_7af3_46b4_bdc3_368a9332dbc2.slice/crio-6471913a2ed9b65d82ff4427b7c6649c3d733f0a9d9bfd967e175d5989f4ca1a WatchSource:0}: Error finding container 6471913a2ed9b65d82ff4427b7c6649c3d733f0a9d9bfd967e175d5989f4ca1a: Status 404 returned error can't find the container with id 6471913a2ed9b65d82ff4427b7c6649c3d733f0a9d9bfd967e175d5989f4ca1a
Oct 30 19:26:51.386598 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:51.386575    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b9b54ffb7-7xdf5"
Oct 30 19:26:51.686570 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:51.686544    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b9b54ffb7-7xdf5"]
Oct 30 19:26:51.688694 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:26:51.688659    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6be54382_a07f_4e83_949e_3bf550c46599.slice/crio-1761863f841944c032f0dfbab5550af9462b03e66237a9d6a969ba86e3e882ba WatchSource:0}: Error finding container 1761863f841944c032f0dfbab5550af9462b03e66237a9d6a969ba86e3e882ba: Status 404 returned error can't find the container with id 1761863f841944c032f0dfbab5550af9462b03e66237a9d6a969ba86e3e882ba
Oct 30 19:26:51.750436 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:51.750399    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-5m84s" event={"ID":"9baa9845-7af3-46b4-bdc3-368a9332dbc2","Type":"ContainerStarted","Data":"6471913a2ed9b65d82ff4427b7c6649c3d733f0a9d9bfd967e175d5989f4ca1a"}
Oct 30 19:26:51.751838 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:51.751812    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b9b54ffb7-7xdf5" event={"ID":"6be54382-a07f-4e83-949e-3bf550c46599","Type":"ContainerStarted","Data":"1761863f841944c032f0dfbab5550af9462b03e66237a9d6a969ba86e3e882ba"}
Oct 30 19:26:54.771828 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:54.771783    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-5m84s" event={"ID":"9baa9845-7af3-46b4-bdc3-368a9332dbc2","Type":"ContainerStarted","Data":"775af311f7331e3fac0ca9cb32b6ae15cc3435f0b988296ae570fa15e7fa4ec9"}
Oct 30 19:26:54.772346 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:54.771854    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-579cf7c6d-5m84s"
Oct 30 19:26:54.773587 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:54.773559    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b9b54ffb7-7xdf5" event={"ID":"6be54382-a07f-4e83-949e-3bf550c46599","Type":"ContainerStarted","Data":"5f24650652fa11ac5c910eeed31d6a994f3f8d5eb6b7da503b3172529bbc46ec"}
Oct 30 19:26:54.773745 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:54.773689    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6b9b54ffb7-7xdf5"
Oct 30 19:26:54.817847 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:54.817795    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-579cf7c6d-5m84s" podStartSLOduration=1.9290506280000002 podStartE2EDuration="4.81778236s" totalImagesPullingTime="2.888731732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:26:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:26:54.816575769 +0000 UTC m=+326.659507722" watchObservedRunningTime="2026-10-30 19:26:54.81778236 +0000 UTC m=+326.660714299"
Oct 30 19:26:54.859700 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:54.859652    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-6b9b54ffb7-7xdf5" podStartSLOduration=1.145417271 podStartE2EDuration="3.859638013s" totalImagesPullingTime="2.714220742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:26:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:26:54.859132994 +0000 UTC m=+326.702064933" watchObservedRunningTime="2026-10-30 19:26:54.859638013 +0000 UTC m=+326.702569952"
Oct 30 19:26:56.780365 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:56.780334    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9wds9"]
Oct 30 19:26:56.784068 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:56.784048    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9wds9"
Oct 30 19:26:56.814462 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:56.814432    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-bfw65\""
Oct 30 19:26:56.814678 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:56.814440    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\""
Oct 30 19:26:56.814678 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:56.814470    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\""
Oct 30 19:26:56.814911 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:56.814549    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\""
Oct 30 19:26:56.816076 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:56.816053    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9wds9"]
Oct 30 19:26:56.842978 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:56.842948    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/4547301a-e9a4-4ecf-8946-55b8ea7c54a9-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-9wds9\" (UID: \"4547301a-e9a4-4ecf-8946-55b8ea7c54a9\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9wds9"
Oct 30 19:26:56.842978 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:56.842980    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w565p\" (UniqueName: \"kubernetes.io/projected/4547301a-e9a4-4ecf-8946-55b8ea7c54a9-kube-api-access-w565p\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-9wds9\" (UID: \"4547301a-e9a4-4ecf-8946-55b8ea7c54a9\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9wds9"
Oct 30 19:26:56.843218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:56.843002    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4547301a-e9a4-4ecf-8946-55b8ea7c54a9-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-9wds9\" (UID: \"4547301a-e9a4-4ecf-8946-55b8ea7c54a9\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9wds9"
Oct 30 19:26:56.843218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:56.843115    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4547301a-e9a4-4ecf-8946-55b8ea7c54a9-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-9wds9\" (UID: \"4547301a-e9a4-4ecf-8946-55b8ea7c54a9\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9wds9"
Oct 30 19:26:56.944206 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:56.944170    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/4547301a-e9a4-4ecf-8946-55b8ea7c54a9-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-9wds9\" (UID: \"4547301a-e9a4-4ecf-8946-55b8ea7c54a9\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9wds9"
Oct 30 19:26:56.944206 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:56.944208    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w565p\" (UniqueName: \"kubernetes.io/projected/4547301a-e9a4-4ecf-8946-55b8ea7c54a9-kube-api-access-w565p\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-9wds9\" (UID: \"4547301a-e9a4-4ecf-8946-55b8ea7c54a9\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9wds9"
Oct 30 19:26:56.944479 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:56.944229    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4547301a-e9a4-4ecf-8946-55b8ea7c54a9-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-9wds9\" (UID: \"4547301a-e9a4-4ecf-8946-55b8ea7c54a9\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9wds9"
Oct 30 19:26:56.944479 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:56.944256    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4547301a-e9a4-4ecf-8946-55b8ea7c54a9-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-9wds9\" (UID: \"4547301a-e9a4-4ecf-8946-55b8ea7c54a9\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9wds9"
Oct 30 19:26:56.944847 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:56.944823    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4547301a-e9a4-4ecf-8946-55b8ea7c54a9-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-9wds9\" (UID: \"4547301a-e9a4-4ecf-8946-55b8ea7c54a9\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9wds9"
Oct 30 19:26:56.945163 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:56.945142    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/4547301a-e9a4-4ecf-8946-55b8ea7c54a9-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-9wds9\" (UID: \"4547301a-e9a4-4ecf-8946-55b8ea7c54a9\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9wds9"
Oct 30 19:26:56.946933 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:56.946912    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4547301a-e9a4-4ecf-8946-55b8ea7c54a9-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-9wds9\" (UID: \"4547301a-e9a4-4ecf-8946-55b8ea7c54a9\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9wds9"
Oct 30 19:26:56.966555 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:56.966523    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w565p\" (UniqueName: \"kubernetes.io/projected/4547301a-e9a4-4ecf-8946-55b8ea7c54a9-kube-api-access-w565p\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-9wds9\" (UID: \"4547301a-e9a4-4ecf-8946-55b8ea7c54a9\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9wds9"
Oct 30 19:26:57.097890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:57.097770    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9wds9"
Oct 30 19:26:57.397946 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:57.397922    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9wds9"]
Oct 30 19:26:57.400388 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:26:57.400359    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4547301a_e9a4_4ecf_8946_55b8ea7c54a9.slice/crio-4e3c47eaa42b3965fc82edb301213708cb7be4139de835ad86fba1f4c19b054b WatchSource:0}: Error finding container 4e3c47eaa42b3965fc82edb301213708cb7be4139de835ad86fba1f4c19b054b: Status 404 returned error can't find the container with id 4e3c47eaa42b3965fc82edb301213708cb7be4139de835ad86fba1f4c19b054b
Oct 30 19:26:57.402580 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:57.402557    5907 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:26:57.788773 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:26:57.788689    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9wds9" event={"ID":"4547301a-e9a4-4ecf-8946-55b8ea7c54a9","Type":"ContainerStarted","Data":"4e3c47eaa42b3965fc82edb301213708cb7be4139de835ad86fba1f4c19b054b"}
Oct 30 19:27:05.780681 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:05.780644    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-579cf7c6d-5m84s"
Oct 30 19:27:05.781164 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:05.780743    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6b9b54ffb7-7xdf5"
Oct 30 19:27:05.904435 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:05.904399    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-5m84s"]
Oct 30 19:27:05.904678 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:05.904624    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-579cf7c6d-5m84s" podUID="9baa9845-7af3-46b4-bdc3-368a9332dbc2" containerName="manager" containerID="cri-o://775af311f7331e3fac0ca9cb32b6ae15cc3435f0b988296ae570fa15e7fa4ec9" gracePeriod=30
Oct 30 19:27:06.165643 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:06.165617    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-5m84s"
Oct 30 19:27:06.348062 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:06.347967    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9baa9845-7af3-46b4-bdc3-368a9332dbc2-webhook-cert\" (UniqueName: \"kubernetes.io/secret/9baa9845-7af3-46b4-bdc3-368a9332dbc2-webhook-cert\") pod \"9baa9845-7af3-46b4-bdc3-368a9332dbc2\" (UID: \"9baa9845-7af3-46b4-bdc3-368a9332dbc2\") "
Oct 30 19:27:06.348062 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:06.348020    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9baa9845-7af3-46b4-bdc3-368a9332dbc2-kube-api-access-vxplz\" (UniqueName: \"kubernetes.io/projected/9baa9845-7af3-46b4-bdc3-368a9332dbc2-kube-api-access-vxplz\") pod \"9baa9845-7af3-46b4-bdc3-368a9332dbc2\" (UID: \"9baa9845-7af3-46b4-bdc3-368a9332dbc2\") "
Oct 30 19:27:06.348296 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:06.348100    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9baa9845-7af3-46b4-bdc3-368a9332dbc2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9baa9845-7af3-46b4-bdc3-368a9332dbc2-metrics-certs\") pod \"9baa9845-7af3-46b4-bdc3-368a9332dbc2\" (UID: \"9baa9845-7af3-46b4-bdc3-368a9332dbc2\") "
Oct 30 19:27:06.350794 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:06.350755    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9baa9845-7af3-46b4-bdc3-368a9332dbc2-metrics-certs" pod "9baa9845-7af3-46b4-bdc3-368a9332dbc2" (UID: "9baa9845-7af3-46b4-bdc3-368a9332dbc2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:27:06.350958 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:06.350811    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9baa9845-7af3-46b4-bdc3-368a9332dbc2-kube-api-access-vxplz" pod "9baa9845-7af3-46b4-bdc3-368a9332dbc2" (UID: "9baa9845-7af3-46b4-bdc3-368a9332dbc2"). InnerVolumeSpecName "kube-api-access-vxplz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:27:06.350958 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:06.350932    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9baa9845-7af3-46b4-bdc3-368a9332dbc2-webhook-cert" pod "9baa9845-7af3-46b4-bdc3-368a9332dbc2" (UID: "9baa9845-7af3-46b4-bdc3-368a9332dbc2"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:27:06.449215 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:06.449177    5907 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9baa9845-7af3-46b4-bdc3-368a9332dbc2-metrics-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:27:06.449215 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:06.449206    5907 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9baa9845-7af3-46b4-bdc3-368a9332dbc2-webhook-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:27:06.449215 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:06.449217    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxplz\" (UniqueName: \"kubernetes.io/projected/9baa9845-7af3-46b4-bdc3-368a9332dbc2-kube-api-access-vxplz\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:27:06.841251 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:06.841205    5907 generic.go:356] "Generic (PLEG): container finished" podID="9baa9845-7af3-46b4-bdc3-368a9332dbc2" containerID="775af311f7331e3fac0ca9cb32b6ae15cc3435f0b988296ae570fa15e7fa4ec9" exitCode=0
Oct 30 19:27:06.841741 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:06.841277    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-5m84s" event={"ID":"9baa9845-7af3-46b4-bdc3-368a9332dbc2","Type":"ContainerDied","Data":"775af311f7331e3fac0ca9cb32b6ae15cc3435f0b988296ae570fa15e7fa4ec9"}
Oct 30 19:27:06.841741 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:06.841329    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-5m84s"
Oct 30 19:27:06.841741 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:06.841339    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-5m84s" event={"ID":"9baa9845-7af3-46b4-bdc3-368a9332dbc2","Type":"ContainerDied","Data":"6471913a2ed9b65d82ff4427b7c6649c3d733f0a9d9bfd967e175d5989f4ca1a"}
Oct 30 19:27:06.841741 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:06.841379    5907 scope.go:122] "RemoveContainer" containerID="775af311f7331e3fac0ca9cb32b6ae15cc3435f0b988296ae570fa15e7fa4ec9"
Oct 30 19:27:06.854343 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:06.854322    5907 scope.go:122] "RemoveContainer" containerID="775af311f7331e3fac0ca9cb32b6ae15cc3435f0b988296ae570fa15e7fa4ec9"
Oct 30 19:27:06.854667 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:27:06.854640    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"775af311f7331e3fac0ca9cb32b6ae15cc3435f0b988296ae570fa15e7fa4ec9\": container with ID starting with 775af311f7331e3fac0ca9cb32b6ae15cc3435f0b988296ae570fa15e7fa4ec9 not found: ID does not exist" containerID="775af311f7331e3fac0ca9cb32b6ae15cc3435f0b988296ae570fa15e7fa4ec9"
Oct 30 19:27:06.854784 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:06.854676    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"775af311f7331e3fac0ca9cb32b6ae15cc3435f0b988296ae570fa15e7fa4ec9"} err="failed to get container status \"775af311f7331e3fac0ca9cb32b6ae15cc3435f0b988296ae570fa15e7fa4ec9\": rpc error: code = NotFound desc = could not find container \"775af311f7331e3fac0ca9cb32b6ae15cc3435f0b988296ae570fa15e7fa4ec9\": container with ID starting with 775af311f7331e3fac0ca9cb32b6ae15cc3435f0b988296ae570fa15e7fa4ec9 not found: ID does not exist"
Oct 30 19:27:06.913479 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:06.913446    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-5m84s"]
Oct 30 19:27:06.930642 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:06.930610    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-5m84s"]
Oct 30 19:27:08.287438 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:08.287360    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9baa9845-7af3-46b4-bdc3-368a9332dbc2" path="/var/lib/kubelet/pods/9baa9845-7af3-46b4-bdc3-368a9332dbc2/volumes"
Oct 30 19:27:08.796450 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:08.796414    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zdlcg"]
Oct 30 19:27:08.796908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:08.796894    5907 cpu_manager.go:414] "containerMap: removing container" podUID="9baa9845-7af3-46b4-bdc3-368a9332dbc2" containerName="manager"
Oct 30 19:27:08.796957 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:08.796910    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9baa9845-7af3-46b4-bdc3-368a9332dbc2" containerName="manager"
Oct 30 19:27:08.797055 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:08.797044    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="9baa9845-7af3-46b4-bdc3-368a9332dbc2" containerName="manager"
Oct 30 19:27:08.805471 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:08.805444    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zdlcg"
Oct 30 19:27:08.824706 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:08.824678    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 30 19:27:08.826419 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:08.826391    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-6vscc\""
Oct 30 19:27:08.826520 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:08.826397    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 30 19:27:08.826520 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:08.826476    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 30 19:27:08.838953 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:08.838926    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 30 19:27:08.841745 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:08.841718    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zdlcg"]
Oct 30 19:27:08.853037 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:08.853008    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9wds9" event={"ID":"4547301a-e9a4-4ecf-8946-55b8ea7c54a9","Type":"ContainerStarted","Data":"967c2dba5717faef9f0c98e667c7bf1e2c95aa51592f0fb539d2e68948dfb4be"}
Oct 30 19:27:08.853167 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:08.853146    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9wds9"
Oct 30 19:27:08.952884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:08.952812    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9wds9" podStartSLOduration=2.463207787 podStartE2EDuration="12.952796964s" totalImagesPullingTime="10.489589177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:26:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:27:08.951934277 +0000 UTC m=+340.794866216" watchObservedRunningTime="2026-10-30 19:27:08.952796964 +0000 UTC m=+340.795728929"
Oct 30 19:27:08.972611 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:08.972573    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8b6d708-4cea-4eec-b78d-030047017ecc-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zdlcg\" (UID: \"b8b6d708-4cea-4eec-b78d-030047017ecc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zdlcg"
Oct 30 19:27:08.972611 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:08.972612    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b8b6d708-4cea-4eec-b78d-030047017ecc-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zdlcg\" (UID: \"b8b6d708-4cea-4eec-b78d-030047017ecc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zdlcg"
Oct 30 19:27:08.972911 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:08.972882    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8b6d708-4cea-4eec-b78d-030047017ecc-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zdlcg\" (UID: \"b8b6d708-4cea-4eec-b78d-030047017ecc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zdlcg"
Oct 30 19:27:08.972969 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:08.972939    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b8b6d708-4cea-4eec-b78d-030047017ecc-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zdlcg\" (UID: \"b8b6d708-4cea-4eec-b78d-030047017ecc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zdlcg"
Oct 30 19:27:08.973108 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:08.973090    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b8b6d708-4cea-4eec-b78d-030047017ecc-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zdlcg\" (UID: \"b8b6d708-4cea-4eec-b78d-030047017ecc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zdlcg"
Oct 30 19:27:08.973168 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:08.973140    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b8b6d708-4cea-4eec-b78d-030047017ecc-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zdlcg\" (UID: \"b8b6d708-4cea-4eec-b78d-030047017ecc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zdlcg"
Oct 30 19:27:08.973275 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:08.973254    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn84s\" (UniqueName: \"kubernetes.io/projected/b8b6d708-4cea-4eec-b78d-030047017ecc-kube-api-access-bn84s\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zdlcg\" (UID: \"b8b6d708-4cea-4eec-b78d-030047017ecc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zdlcg"
Oct 30 19:27:09.074366 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:09.074276    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b8b6d708-4cea-4eec-b78d-030047017ecc-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zdlcg\" (UID: \"b8b6d708-4cea-4eec-b78d-030047017ecc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zdlcg"
Oct 30 19:27:09.074366 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:09.074327    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b8b6d708-4cea-4eec-b78d-030047017ecc-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zdlcg\" (UID: \"b8b6d708-4cea-4eec-b78d-030047017ecc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zdlcg"
Oct 30 19:27:09.074579 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:09.074383    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn84s\" (UniqueName: \"kubernetes.io/projected/b8b6d708-4cea-4eec-b78d-030047017ecc-kube-api-access-bn84s\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zdlcg\" (UID: \"b8b6d708-4cea-4eec-b78d-030047017ecc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zdlcg"
Oct 30 19:27:09.074579 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:09.074433    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8b6d708-4cea-4eec-b78d-030047017ecc-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zdlcg\" (UID: \"b8b6d708-4cea-4eec-b78d-030047017ecc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zdlcg"
Oct 30 19:27:09.074579 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:09.074479    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b8b6d708-4cea-4eec-b78d-030047017ecc-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zdlcg\" (UID: \"b8b6d708-4cea-4eec-b78d-030047017ecc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zdlcg"
Oct 30 19:27:09.074579 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:09.074513    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8b6d708-4cea-4eec-b78d-030047017ecc-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zdlcg\" (UID: \"b8b6d708-4cea-4eec-b78d-030047017ecc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zdlcg"
Oct 30 19:27:09.074579 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:09.074544    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b8b6d708-4cea-4eec-b78d-030047017ecc-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zdlcg\" (UID: \"b8b6d708-4cea-4eec-b78d-030047017ecc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zdlcg"
Oct 30 19:27:09.075563 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:09.075536    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b8b6d708-4cea-4eec-b78d-030047017ecc-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zdlcg\" (UID: \"b8b6d708-4cea-4eec-b78d-030047017ecc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zdlcg"
Oct 30 19:27:09.076830 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:09.076809    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8b6d708-4cea-4eec-b78d-030047017ecc-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zdlcg\" (UID: \"b8b6d708-4cea-4eec-b78d-030047017ecc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zdlcg"
Oct 30 19:27:09.077091 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:09.077069    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b8b6d708-4cea-4eec-b78d-030047017ecc-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zdlcg\" (UID: \"b8b6d708-4cea-4eec-b78d-030047017ecc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zdlcg"
Oct 30 19:27:09.077135 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:09.077078    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b8b6d708-4cea-4eec-b78d-030047017ecc-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zdlcg\" (UID: \"b8b6d708-4cea-4eec-b78d-030047017ecc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zdlcg"
Oct 30 19:27:09.077170 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:09.077128    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b8b6d708-4cea-4eec-b78d-030047017ecc-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zdlcg\" (UID: \"b8b6d708-4cea-4eec-b78d-030047017ecc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zdlcg"
Oct 30 19:27:09.095705 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:09.095676    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8b6d708-4cea-4eec-b78d-030047017ecc-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zdlcg\" (UID: \"b8b6d708-4cea-4eec-b78d-030047017ecc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zdlcg"
Oct 30 19:27:09.095945 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:09.095927    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn84s\" (UniqueName: \"kubernetes.io/projected/b8b6d708-4cea-4eec-b78d-030047017ecc-kube-api-access-bn84s\") pod \"istiod-openshift-gateway-7ddd9cfdc8-zdlcg\" (UID: \"b8b6d708-4cea-4eec-b78d-030047017ecc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zdlcg"
Oct 30 19:27:09.117568 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:09.117545    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zdlcg"
Oct 30 19:27:09.438574 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:09.438550    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zdlcg"]
Oct 30 19:27:09.440566 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:27:09.440533    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8b6d708_4cea_4eec_b78d_030047017ecc.slice/crio-40849abe0d4f0d6616ab22c4600f44383d473f63da90c2dd3880a695759da0e7 WatchSource:0}: Error finding container 40849abe0d4f0d6616ab22c4600f44383d473f63da90c2dd3880a695759da0e7: Status 404 returned error can't find the container with id 40849abe0d4f0d6616ab22c4600f44383d473f63da90c2dd3880a695759da0e7
Oct 30 19:27:09.859274 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:09.859190    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zdlcg" event={"ID":"b8b6d708-4cea-4eec-b78d-030047017ecc","Type":"ContainerStarted","Data":"40849abe0d4f0d6616ab22c4600f44383d473f63da90c2dd3880a695759da0e7"}
Oct 30 19:27:17.509105 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:27:17.509061    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:27:18.187111 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:18.187079    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:27:18.187183 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:18.187147    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:27:18.904055 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:18.904015    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zdlcg" event={"ID":"b8b6d708-4cea-4eec-b78d-030047017ecc","Type":"ContainerStarted","Data":"06612e82a99c6212a11fad3a0a924efc3229197e0c01ead941d09837b0ef1feb"}
Oct 30 19:27:18.904444 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:18.904138    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zdlcg"
Oct 30 19:27:18.976381 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:18.976324    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zdlcg" podStartSLOduration=2.232759851 podStartE2EDuration="10.976309164s" totalImagesPullingTime="8.743549313s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:27:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:27:18.962134066 +0000 UTC m=+350.805066007" watchObservedRunningTime="2026-10-30 19:27:18.976309164 +0000 UTC m=+350.819241103"
Oct 30 19:27:19.864806 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:19.864776    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9wds9"
Oct 30 19:27:19.910605 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:19.910580    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zdlcg"
Oct 30 19:27:29.903812 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:27:29.903771    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:27:47.499011 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:27:47.498926    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:28:17.499283 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:28:17.499245    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:28:47.502028 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:28:47.501988    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:29:17.498805 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:29:17.498763    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:29:19.885344 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:19.885305    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:29:24.651410 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:24.651372    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:29:30.197801 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:30.197765    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp"]
Oct 30 19:29:30.201527 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:30.201502    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp"
Oct 30 19:29:30.226715 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:30.226682    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-tlm72\""
Oct 30 19:29:30.230815 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:30.230791    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Oct 30 19:29:30.230815 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:30.230807    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Oct 30 19:29:30.231007 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:30.230946    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Oct 30 19:29:30.232517 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:30.232496    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp"]
Oct 30 19:29:30.348685 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:30.348656    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/817de1af-900b-4fbe-b792-24ac2de8b171-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp\" (UID: \"817de1af-900b-4fbe-b792-24ac2de8b171\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp"
Oct 30 19:29:30.348912 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:30.348693    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/817de1af-900b-4fbe-b792-24ac2de8b171-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp\" (UID: \"817de1af-900b-4fbe-b792-24ac2de8b171\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp"
Oct 30 19:29:30.348912 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:30.348816    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/817de1af-900b-4fbe-b792-24ac2de8b171-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp\" (UID: \"817de1af-900b-4fbe-b792-24ac2de8b171\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp"
Oct 30 19:29:30.349084 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:30.349067    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/817de1af-900b-4fbe-b792-24ac2de8b171-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp\" (UID: \"817de1af-900b-4fbe-b792-24ac2de8b171\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp"
Oct 30 19:29:30.349168 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:30.349148    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/817de1af-900b-4fbe-b792-24ac2de8b171-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp\" (UID: \"817de1af-900b-4fbe-b792-24ac2de8b171\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp"
Oct 30 19:29:30.349357 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:30.349338    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nj7ps\" (UniqueName: \"kubernetes.io/projected/817de1af-900b-4fbe-b792-24ac2de8b171-kube-api-access-nj7ps\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp\" (UID: \"817de1af-900b-4fbe-b792-24ac2de8b171\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp"
Oct 30 19:29:30.450711 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:30.450622    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/817de1af-900b-4fbe-b792-24ac2de8b171-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp\" (UID: \"817de1af-900b-4fbe-b792-24ac2de8b171\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp"
Oct 30 19:29:30.450711 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:30.450668    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/817de1af-900b-4fbe-b792-24ac2de8b171-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp\" (UID: \"817de1af-900b-4fbe-b792-24ac2de8b171\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp"
Oct 30 19:29:30.450711 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:30.450697    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/817de1af-900b-4fbe-b792-24ac2de8b171-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp\" (UID: \"817de1af-900b-4fbe-b792-24ac2de8b171\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp"
Oct 30 19:29:30.451050 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:30.450736    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/817de1af-900b-4fbe-b792-24ac2de8b171-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp\" (UID: \"817de1af-900b-4fbe-b792-24ac2de8b171\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp"
Oct 30 19:29:30.451050 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:30.450780    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/817de1af-900b-4fbe-b792-24ac2de8b171-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp\" (UID: \"817de1af-900b-4fbe-b792-24ac2de8b171\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp"
Oct 30 19:29:30.451050 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:30.450828    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nj7ps\" (UniqueName: \"kubernetes.io/projected/817de1af-900b-4fbe-b792-24ac2de8b171-kube-api-access-nj7ps\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp\" (UID: \"817de1af-900b-4fbe-b792-24ac2de8b171\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp"
Oct 30 19:29:30.451208 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:30.451170    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/817de1af-900b-4fbe-b792-24ac2de8b171-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp\" (UID: \"817de1af-900b-4fbe-b792-24ac2de8b171\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp"
Oct 30 19:29:30.451255 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:30.451203    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/817de1af-900b-4fbe-b792-24ac2de8b171-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp\" (UID: \"817de1af-900b-4fbe-b792-24ac2de8b171\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp"
Oct 30 19:29:30.451289 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:30.451262    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/817de1af-900b-4fbe-b792-24ac2de8b171-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp\" (UID: \"817de1af-900b-4fbe-b792-24ac2de8b171\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp"
Oct 30 19:29:30.453314 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:30.453287    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/817de1af-900b-4fbe-b792-24ac2de8b171-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp\" (UID: \"817de1af-900b-4fbe-b792-24ac2de8b171\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp"
Oct 30 19:29:30.453561 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:30.453544    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/817de1af-900b-4fbe-b792-24ac2de8b171-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp\" (UID: \"817de1af-900b-4fbe-b792-24ac2de8b171\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp"
Oct 30 19:29:30.472109 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:30.472079    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nj7ps\" (UniqueName: \"kubernetes.io/projected/817de1af-900b-4fbe-b792-24ac2de8b171-kube-api-access-nj7ps\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp\" (UID: \"817de1af-900b-4fbe-b792-24ac2de8b171\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp"
Oct 30 19:29:30.518156 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:30.518123    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp"
Oct 30 19:29:30.821691 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:30.821489    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp"]
Oct 30 19:29:30.824518 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:29:30.824485    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod817de1af_900b_4fbe_b792_24ac2de8b171.slice/crio-530fb647cd71d5efdf9161c624e46ce96648105f79804616869c577ca70dfcea WatchSource:0}: Error finding container 530fb647cd71d5efdf9161c624e46ce96648105f79804616869c577ca70dfcea: Status 404 returned error can't find the container with id 530fb647cd71d5efdf9161c624e46ce96648105f79804616869c577ca70dfcea
Oct 30 19:29:31.508092 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:31.508055    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp" event={"ID":"817de1af-900b-4fbe-b792-24ac2de8b171","Type":"ContainerStarted","Data":"530fb647cd71d5efdf9161c624e46ce96648105f79804616869c577ca70dfcea"}
Oct 30 19:29:31.592951 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:31.592915    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:29:32.514696 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:32.514657    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp" event={"ID":"817de1af-900b-4fbe-b792-24ac2de8b171","Type":"ContainerStarted","Data":"acdccbbbcc0a9bf65d72d0893a56c9c6721728db4c262664d367a7c477bd1e16"}
Oct 30 19:29:32.519060 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:32.519041    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp"
Oct 30 19:29:32.558803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:32.558717    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp" podStartSLOduration=1.526560009 podStartE2EDuration="2.558699835s" totalImagesPullingTime="1.032139826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:29:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:29:32.558183289 +0000 UTC m=+484.401115220" watchObservedRunningTime="2026-10-30 19:29:32.558699835 +0000 UTC m=+484.401631775"
Oct 30 19:29:33.412337 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:33.412304    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-wd6m6"]
Oct 30 19:29:33.416065 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:33.416046    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wd6m6"
Oct 30 19:29:33.431957 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:33.431925    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Oct 30 19:29:33.446518 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:33.446487    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-wd6m6"]
Oct 30 19:29:33.584585 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:33.584544    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b52bf40-f75b-465b-8947-10ea73874f85-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wd6m6\" (UID: \"0b52bf40-f75b-465b-8947-10ea73874f85\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wd6m6"
Oct 30 19:29:33.585005 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:33.584680    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcz2w\" (UniqueName: \"kubernetes.io/projected/0b52bf40-f75b-465b-8947-10ea73874f85-kube-api-access-jcz2w\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wd6m6\" (UID: \"0b52bf40-f75b-465b-8947-10ea73874f85\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wd6m6"
Oct 30 19:29:33.585005 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:33.584748    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b52bf40-f75b-465b-8947-10ea73874f85-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wd6m6\" (UID: \"0b52bf40-f75b-465b-8947-10ea73874f85\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wd6m6"
Oct 30 19:29:33.585005 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:33.584770    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b52bf40-f75b-465b-8947-10ea73874f85-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wd6m6\" (UID: \"0b52bf40-f75b-465b-8947-10ea73874f85\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wd6m6"
Oct 30 19:29:33.585005 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:33.584846    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b52bf40-f75b-465b-8947-10ea73874f85-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wd6m6\" (UID: \"0b52bf40-f75b-465b-8947-10ea73874f85\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wd6m6"
Oct 30 19:29:33.585005 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:33.584904    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b52bf40-f75b-465b-8947-10ea73874f85-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wd6m6\" (UID: \"0b52bf40-f75b-465b-8947-10ea73874f85\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wd6m6"
Oct 30 19:29:33.686238 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:33.686154    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b52bf40-f75b-465b-8947-10ea73874f85-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wd6m6\" (UID: \"0b52bf40-f75b-465b-8947-10ea73874f85\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wd6m6"
Oct 30 19:29:33.686238 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:33.686192    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b52bf40-f75b-465b-8947-10ea73874f85-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wd6m6\" (UID: \"0b52bf40-f75b-465b-8947-10ea73874f85\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wd6m6"
Oct 30 19:29:33.686238 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:33.686212    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b52bf40-f75b-465b-8947-10ea73874f85-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wd6m6\" (UID: \"0b52bf40-f75b-465b-8947-10ea73874f85\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wd6m6"
Oct 30 19:29:33.686238 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:33.686227    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b52bf40-f75b-465b-8947-10ea73874f85-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wd6m6\" (UID: \"0b52bf40-f75b-465b-8947-10ea73874f85\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wd6m6"
Oct 30 19:29:33.686566 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:33.686379    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b52bf40-f75b-465b-8947-10ea73874f85-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wd6m6\" (UID: \"0b52bf40-f75b-465b-8947-10ea73874f85\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wd6m6"
Oct 30 19:29:33.686566 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:33.686526    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcz2w\" (UniqueName: \"kubernetes.io/projected/0b52bf40-f75b-465b-8947-10ea73874f85-kube-api-access-jcz2w\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wd6m6\" (UID: \"0b52bf40-f75b-465b-8947-10ea73874f85\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wd6m6"
Oct 30 19:29:33.686673 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:33.686643    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b52bf40-f75b-465b-8947-10ea73874f85-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wd6m6\" (UID: \"0b52bf40-f75b-465b-8947-10ea73874f85\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wd6m6"
Oct 30 19:29:33.686734 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:33.686697    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b52bf40-f75b-465b-8947-10ea73874f85-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wd6m6\" (UID: \"0b52bf40-f75b-465b-8947-10ea73874f85\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wd6m6"
Oct 30 19:29:33.686887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:33.686837    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b52bf40-f75b-465b-8947-10ea73874f85-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wd6m6\" (UID: \"0b52bf40-f75b-465b-8947-10ea73874f85\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wd6m6"
Oct 30 19:29:33.688611 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:33.688591    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b52bf40-f75b-465b-8947-10ea73874f85-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wd6m6\" (UID: \"0b52bf40-f75b-465b-8947-10ea73874f85\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wd6m6"
Oct 30 19:29:33.688875 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:33.688844    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b52bf40-f75b-465b-8947-10ea73874f85-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wd6m6\" (UID: \"0b52bf40-f75b-465b-8947-10ea73874f85\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wd6m6"
Oct 30 19:29:33.707029 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:33.707005    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcz2w\" (UniqueName: \"kubernetes.io/projected/0b52bf40-f75b-465b-8947-10ea73874f85-kube-api-access-jcz2w\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-wd6m6\" (UID: \"0b52bf40-f75b-465b-8947-10ea73874f85\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wd6m6"
Oct 30 19:29:33.732637 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:33.732608    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wd6m6"
Oct 30 19:29:34.032387 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:34.032307    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-wd6m6"]
Oct 30 19:29:34.034623 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:29:34.034598    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b52bf40_f75b_465b_8947_10ea73874f85.slice/crio-b4a6fc8684adfc8df22899623567e1821dbc95a4673e2948a8f148a1e19575ff WatchSource:0}: Error finding container b4a6fc8684adfc8df22899623567e1821dbc95a4673e2948a8f148a1e19575ff: Status 404 returned error can't find the container with id b4a6fc8684adfc8df22899623567e1821dbc95a4673e2948a8f148a1e19575ff
Oct 30 19:29:34.525207 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:34.525172    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wd6m6" event={"ID":"0b52bf40-f75b-465b-8947-10ea73874f85","Type":"ContainerStarted","Data":"8c8419852ebf3785826e28ea163d5c1f7482eee85f67719d54204124b3cc5298"}
Oct 30 19:29:34.525402 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:34.525228    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wd6m6" event={"ID":"0b52bf40-f75b-465b-8947-10ea73874f85","Type":"ContainerStarted","Data":"b4a6fc8684adfc8df22899623567e1821dbc95a4673e2948a8f148a1e19575ff"}
Oct 30 19:29:34.569941 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:34.569789    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wd6m6" podStartSLOduration=1.361935402 podStartE2EDuration="1.569768162s" totalImagesPullingTime="207.83276ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:29:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:29:34.56925661 +0000 UTC m=+486.412188552" watchObservedRunningTime="2026-10-30 19:29:34.569768162 +0000 UTC m=+486.412700102"
Oct 30 19:29:34.733572 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:34.733527    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wd6m6"
Oct 30 19:29:38.260152 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:38.260111    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"]
Oct 30 19:29:38.265556 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:38.265522    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"
Oct 30 19:29:38.288454 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:38.288414    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"]
Oct 30 19:29:38.361132 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:38.361101    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-q4g2l\""
Oct 30 19:29:38.434930 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:38.434895    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8db50859-98fc-49aa-806b-4fbed56b1d00-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9tdql\" (UID: \"8db50859-98fc-49aa-806b-4fbed56b1d00\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"
Oct 30 19:29:38.434930 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:38.434940    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hczlz\" (UniqueName: \"kubernetes.io/projected/8db50859-98fc-49aa-806b-4fbed56b1d00-kube-api-access-hczlz\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9tdql\" (UID: \"8db50859-98fc-49aa-806b-4fbed56b1d00\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"
Oct 30 19:29:38.435163 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:38.435128    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8db50859-98fc-49aa-806b-4fbed56b1d00-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9tdql\" (UID: \"8db50859-98fc-49aa-806b-4fbed56b1d00\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"
Oct 30 19:29:38.435238 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:38.435223    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8db50859-98fc-49aa-806b-4fbed56b1d00-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9tdql\" (UID: \"8db50859-98fc-49aa-806b-4fbed56b1d00\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"
Oct 30 19:29:38.435303 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:38.435271    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8db50859-98fc-49aa-806b-4fbed56b1d00-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9tdql\" (UID: \"8db50859-98fc-49aa-806b-4fbed56b1d00\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"
Oct 30 19:29:38.435374 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:38.435355    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8db50859-98fc-49aa-806b-4fbed56b1d00-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9tdql\" (UID: \"8db50859-98fc-49aa-806b-4fbed56b1d00\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"
Oct 30 19:29:38.435469 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:38.435447    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8db50859-98fc-49aa-806b-4fbed56b1d00-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9tdql\" (UID: \"8db50859-98fc-49aa-806b-4fbed56b1d00\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"
Oct 30 19:29:38.435548 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:38.435533    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8db50859-98fc-49aa-806b-4fbed56b1d00-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9tdql\" (UID: \"8db50859-98fc-49aa-806b-4fbed56b1d00\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"
Oct 30 19:29:38.435669 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:38.435649    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8db50859-98fc-49aa-806b-4fbed56b1d00-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9tdql\" (UID: \"8db50859-98fc-49aa-806b-4fbed56b1d00\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"
Oct 30 19:29:38.536345 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:38.536269    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8db50859-98fc-49aa-806b-4fbed56b1d00-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9tdql\" (UID: \"8db50859-98fc-49aa-806b-4fbed56b1d00\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"
Oct 30 19:29:38.536345 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:38.536314    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8db50859-98fc-49aa-806b-4fbed56b1d00-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9tdql\" (UID: \"8db50859-98fc-49aa-806b-4fbed56b1d00\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"
Oct 30 19:29:38.536345 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:38.536346    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hczlz\" (UniqueName: \"kubernetes.io/projected/8db50859-98fc-49aa-806b-4fbed56b1d00-kube-api-access-hczlz\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9tdql\" (UID: \"8db50859-98fc-49aa-806b-4fbed56b1d00\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"
Oct 30 19:29:38.536608 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:38.536374    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8db50859-98fc-49aa-806b-4fbed56b1d00-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9tdql\" (UID: \"8db50859-98fc-49aa-806b-4fbed56b1d00\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"
Oct 30 19:29:38.536608 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:38.536412    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8db50859-98fc-49aa-806b-4fbed56b1d00-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9tdql\" (UID: \"8db50859-98fc-49aa-806b-4fbed56b1d00\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"
Oct 30 19:29:38.536608 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:38.536432    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8db50859-98fc-49aa-806b-4fbed56b1d00-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9tdql\" (UID: \"8db50859-98fc-49aa-806b-4fbed56b1d00\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"
Oct 30 19:29:38.536761 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:38.536604    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8db50859-98fc-49aa-806b-4fbed56b1d00-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9tdql\" (UID: \"8db50859-98fc-49aa-806b-4fbed56b1d00\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"
Oct 30 19:29:38.536761 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:38.536664    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8db50859-98fc-49aa-806b-4fbed56b1d00-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9tdql\" (UID: \"8db50859-98fc-49aa-806b-4fbed56b1d00\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"
Oct 30 19:29:38.536761 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:38.536701    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8db50859-98fc-49aa-806b-4fbed56b1d00-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9tdql\" (UID: \"8db50859-98fc-49aa-806b-4fbed56b1d00\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"
Oct 30 19:29:38.536761 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:38.536712    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8db50859-98fc-49aa-806b-4fbed56b1d00-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9tdql\" (UID: \"8db50859-98fc-49aa-806b-4fbed56b1d00\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"
Oct 30 19:29:38.537138 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:38.537122    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8db50859-98fc-49aa-806b-4fbed56b1d00-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9tdql\" (UID: \"8db50859-98fc-49aa-806b-4fbed56b1d00\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"
Oct 30 19:29:38.537242 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:38.537185    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8db50859-98fc-49aa-806b-4fbed56b1d00-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9tdql\" (UID: \"8db50859-98fc-49aa-806b-4fbed56b1d00\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"
Oct 30 19:29:38.537303 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:38.537237    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8db50859-98fc-49aa-806b-4fbed56b1d00-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9tdql\" (UID: \"8db50859-98fc-49aa-806b-4fbed56b1d00\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"
Oct 30 19:29:38.537303 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:38.537238    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8db50859-98fc-49aa-806b-4fbed56b1d00-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9tdql\" (UID: \"8db50859-98fc-49aa-806b-4fbed56b1d00\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"
Oct 30 19:29:38.539132 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:38.539106    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8db50859-98fc-49aa-806b-4fbed56b1d00-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9tdql\" (UID: \"8db50859-98fc-49aa-806b-4fbed56b1d00\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"
Oct 30 19:29:38.539511 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:38.539485    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8db50859-98fc-49aa-806b-4fbed56b1d00-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9tdql\" (UID: \"8db50859-98fc-49aa-806b-4fbed56b1d00\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"
Oct 30 19:29:38.557706 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:38.557688    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8db50859-98fc-49aa-806b-4fbed56b1d00-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9tdql\" (UID: \"8db50859-98fc-49aa-806b-4fbed56b1d00\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"
Oct 30 19:29:38.557829 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:38.557741    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hczlz\" (UniqueName: \"kubernetes.io/projected/8db50859-98fc-49aa-806b-4fbed56b1d00-kube-api-access-hczlz\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9tdql\" (UID: \"8db50859-98fc-49aa-806b-4fbed56b1d00\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"
Oct 30 19:29:38.580689 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:38.580665    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"
Oct 30 19:29:38.900212 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:38.900186    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"]
Oct 30 19:29:38.902593 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:29:38.902554    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8db50859_98fc_49aa_806b_4fbed56b1d00.slice/crio-961eb5eaeabbd8cf0d540fcdd7e08dc4c75556f2e4b4700e59a63d24fa75dc56 WatchSource:0}: Error finding container 961eb5eaeabbd8cf0d540fcdd7e08dc4c75556f2e4b4700e59a63d24fa75dc56: Status 404 returned error can't find the container with id 961eb5eaeabbd8cf0d540fcdd7e08dc4c75556f2e4b4700e59a63d24fa75dc56
Oct 30 19:29:38.907795 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:38.907773    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:29:39.550203 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:39.550167    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql" event={"ID":"8db50859-98fc-49aa-806b-4fbed56b1d00","Type":"ContainerStarted","Data":"961eb5eaeabbd8cf0d540fcdd7e08dc4c75556f2e4b4700e59a63d24fa75dc56"}
Oct 30 19:29:40.519434 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:40.519402    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp"
Oct 30 19:29:40.532732 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:40.532703    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp"
Oct 30 19:29:40.569582 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:40.569549    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp"
Oct 30 19:29:41.563703 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:41.563669    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:29:41.563850 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:41.563735    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:29:41.563850 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:41.563762    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:29:42.568635 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:42.568597    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql" event={"ID":"8db50859-98fc-49aa-806b-4fbed56b1d00","Type":"ContainerStarted","Data":"630582f404a4380819b5d48f32dbf4a67a8064d9048659570cf0558b28ded57a"}
Oct 30 19:29:42.581090 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:42.581061    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"
Oct 30 19:29:42.582080 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:42.582058    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-9tdql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Oct 30 19:29:42.582184 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:42.582100    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql" podUID="8db50859-98fc-49aa-806b-4fbed56b1d00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 30 19:29:42.642134 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:42.642078    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql" podStartSLOduration=1.983837456 podStartE2EDuration="4.642062297s" totalImagesPullingTime="2.658224841s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:29:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:29:42.640640807 +0000 UTC m=+494.483572747" watchObservedRunningTime="2026-10-30 19:29:42.642062297 +0000 UTC m=+494.484994236"
Oct 30 19:29:43.581323 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:43.581288    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-9tdql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Oct 30 19:29:43.581723 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:43.581350    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql" podUID="8db50859-98fc-49aa-806b-4fbed56b1d00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 30 19:29:43.735796 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:43.734941    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wd6m6"
Oct 30 19:29:43.747302 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:43.747276    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wd6m6"
Oct 30 19:29:44.581347 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:44.581310    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-9tdql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Oct 30 19:29:44.581743 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:44.581359    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql" podUID="8db50859-98fc-49aa-806b-4fbed56b1d00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 30 19:29:44.589005 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:44.588979    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wd6m6"
Oct 30 19:29:45.581010 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:45.580972    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-9tdql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Oct 30 19:29:45.581211 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:45.581033    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql" podUID="8db50859-98fc-49aa-806b-4fbed56b1d00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 30 19:29:46.581209 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:46.581168    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-9tdql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Oct 30 19:29:46.581613 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:46.581230    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql" podUID="8db50859-98fc-49aa-806b-4fbed56b1d00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 30 19:29:47.499948 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:29:47.499913    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:29:47.581405 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:47.581372    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-9tdql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Oct 30 19:29:47.581940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:47.581424    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql" podUID="8db50859-98fc-49aa-806b-4fbed56b1d00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 30 19:29:48.581579 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:48.581542    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-9tdql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Oct 30 19:29:48.582081 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:48.581599    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql" podUID="8db50859-98fc-49aa-806b-4fbed56b1d00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 30 19:29:49.799563 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:49.799527    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"
Oct 30 19:29:49.800098 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:49.799930    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"
Oct 30 19:29:49.800898 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:49.800879    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql"
Oct 30 19:29:52.161675 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:52.161641    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5f78687d5b-d59pz"]
Oct 30 19:29:52.165109 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:52.165091    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5f78687d5b-d59pz"
Oct 30 19:29:52.187721 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:52.187682    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5f78687d5b-d59pz"]
Oct 30 19:29:52.195274 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:52.195249    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Oct 30 19:29:52.211601 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:52.211576    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-l54cm\""
Oct 30 19:29:52.299315 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:52.299286    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgnn8\" (UniqueName: \"kubernetes.io/projected/91ee2b9c-4412-415e-817d-d92df100c950-kube-api-access-vgnn8\") pod \"payload-processing-5f78687d5b-d59pz\" (UID: \"91ee2b9c-4412-415e-817d-d92df100c950\") " pod="openshift-ingress/payload-processing-5f78687d5b-d59pz"
Oct 30 19:29:52.299509 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:52.299326    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91ee2b9c-4412-415e-817d-d92df100c950-plugins-config-volume\") pod \"payload-processing-5f78687d5b-d59pz\" (UID: \"91ee2b9c-4412-415e-817d-d92df100c950\") " pod="openshift-ingress/payload-processing-5f78687d5b-d59pz"
Oct 30 19:29:52.400887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:52.400831    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91ee2b9c-4412-415e-817d-d92df100c950-plugins-config-volume\") pod \"payload-processing-5f78687d5b-d59pz\" (UID: \"91ee2b9c-4412-415e-817d-d92df100c950\") " pod="openshift-ingress/payload-processing-5f78687d5b-d59pz"
Oct 30 19:29:52.401093 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:52.401062    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgnn8\" (UniqueName: \"kubernetes.io/projected/91ee2b9c-4412-415e-817d-d92df100c950-kube-api-access-vgnn8\") pod \"payload-processing-5f78687d5b-d59pz\" (UID: \"91ee2b9c-4412-415e-817d-d92df100c950\") " pod="openshift-ingress/payload-processing-5f78687d5b-d59pz"
Oct 30 19:29:52.401848 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:52.401821    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91ee2b9c-4412-415e-817d-d92df100c950-plugins-config-volume\") pod \"payload-processing-5f78687d5b-d59pz\" (UID: \"91ee2b9c-4412-415e-817d-d92df100c950\") " pod="openshift-ingress/payload-processing-5f78687d5b-d59pz"
Oct 30 19:29:52.422490 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:52.422417    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgnn8\" (UniqueName: \"kubernetes.io/projected/91ee2b9c-4412-415e-817d-d92df100c950-kube-api-access-vgnn8\") pod \"payload-processing-5f78687d5b-d59pz\" (UID: \"91ee2b9c-4412-415e-817d-d92df100c950\") " pod="openshift-ingress/payload-processing-5f78687d5b-d59pz"
Oct 30 19:29:52.479332 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:52.479297    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5f78687d5b-d59pz"
Oct 30 19:29:52.783366 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:52.783332    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5f78687d5b-d59pz"]
Oct 30 19:29:52.786089 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:29:52.786060    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91ee2b9c_4412_415e_817d_d92df100c950.slice/crio-8c1cbbf26cec5b819801d41c0b92e00d72561a9ba1c056d8d90f1ecf5f50935e WatchSource:0}: Error finding container 8c1cbbf26cec5b819801d41c0b92e00d72561a9ba1c056d8d90f1ecf5f50935e: Status 404 returned error can't find the container with id 8c1cbbf26cec5b819801d41c0b92e00d72561a9ba1c056d8d90f1ecf5f50935e
Oct 30 19:29:53.326243 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.324408    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"]
Oct 30 19:29:53.328384 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.328359    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"
Oct 30 19:29:53.364696 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.364667    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"]
Oct 30 19:29:53.413133 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.413087    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86efaf93-7f28-4e6a-9add-c8bdd87c5261-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-s826m\" (UID: \"86efaf93-7f28-4e6a-9add-c8bdd87c5261\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"
Oct 30 19:29:53.413318 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.413154    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86efaf93-7f28-4e6a-9add-c8bdd87c5261-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-s826m\" (UID: \"86efaf93-7f28-4e6a-9add-c8bdd87c5261\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"
Oct 30 19:29:53.413318 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.413184    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86efaf93-7f28-4e6a-9add-c8bdd87c5261-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-s826m\" (UID: \"86efaf93-7f28-4e6a-9add-c8bdd87c5261\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"
Oct 30 19:29:53.413318 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.413247    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86efaf93-7f28-4e6a-9add-c8bdd87c5261-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-s826m\" (UID: \"86efaf93-7f28-4e6a-9add-c8bdd87c5261\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"
Oct 30 19:29:53.413318 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.413274    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86efaf93-7f28-4e6a-9add-c8bdd87c5261-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-s826m\" (UID: \"86efaf93-7f28-4e6a-9add-c8bdd87c5261\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"
Oct 30 19:29:53.413552 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.413334    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86efaf93-7f28-4e6a-9add-c8bdd87c5261-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-s826m\" (UID: \"86efaf93-7f28-4e6a-9add-c8bdd87c5261\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"
Oct 30 19:29:53.413552 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.413391    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86efaf93-7f28-4e6a-9add-c8bdd87c5261-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-s826m\" (UID: \"86efaf93-7f28-4e6a-9add-c8bdd87c5261\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"
Oct 30 19:29:53.413552 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.413436    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86efaf93-7f28-4e6a-9add-c8bdd87c5261-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-s826m\" (UID: \"86efaf93-7f28-4e6a-9add-c8bdd87c5261\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"
Oct 30 19:29:53.413552 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.413478    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrjl5\" (UniqueName: \"kubernetes.io/projected/86efaf93-7f28-4e6a-9add-c8bdd87c5261-kube-api-access-zrjl5\") pod \"maas-default-gateway-openshift-default-5bc884dd87-s826m\" (UID: \"86efaf93-7f28-4e6a-9add-c8bdd87c5261\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"
Oct 30 19:29:53.514372 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.514334    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86efaf93-7f28-4e6a-9add-c8bdd87c5261-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-s826m\" (UID: \"86efaf93-7f28-4e6a-9add-c8bdd87c5261\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"
Oct 30 19:29:53.514560 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.514394    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86efaf93-7f28-4e6a-9add-c8bdd87c5261-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-s826m\" (UID: \"86efaf93-7f28-4e6a-9add-c8bdd87c5261\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"
Oct 30 19:29:53.514560 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.514451    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86efaf93-7f28-4e6a-9add-c8bdd87c5261-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-s826m\" (UID: \"86efaf93-7f28-4e6a-9add-c8bdd87c5261\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"
Oct 30 19:29:53.514560 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.514509    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86efaf93-7f28-4e6a-9add-c8bdd87c5261-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-s826m\" (UID: \"86efaf93-7f28-4e6a-9add-c8bdd87c5261\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"
Oct 30 19:29:53.514560 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.514542    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86efaf93-7f28-4e6a-9add-c8bdd87c5261-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-s826m\" (UID: \"86efaf93-7f28-4e6a-9add-c8bdd87c5261\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"
Oct 30 19:29:53.514885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.514580    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrjl5\" (UniqueName: \"kubernetes.io/projected/86efaf93-7f28-4e6a-9add-c8bdd87c5261-kube-api-access-zrjl5\") pod \"maas-default-gateway-openshift-default-5bc884dd87-s826m\" (UID: \"86efaf93-7f28-4e6a-9add-c8bdd87c5261\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"
Oct 30 19:29:53.514885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.514623    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86efaf93-7f28-4e6a-9add-c8bdd87c5261-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-s826m\" (UID: \"86efaf93-7f28-4e6a-9add-c8bdd87c5261\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"
Oct 30 19:29:53.514885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.514661    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86efaf93-7f28-4e6a-9add-c8bdd87c5261-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-s826m\" (UID: \"86efaf93-7f28-4e6a-9add-c8bdd87c5261\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"
Oct 30 19:29:53.514885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.514703    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86efaf93-7f28-4e6a-9add-c8bdd87c5261-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-s826m\" (UID: \"86efaf93-7f28-4e6a-9add-c8bdd87c5261\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"
Oct 30 19:29:53.514885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.514874    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86efaf93-7f28-4e6a-9add-c8bdd87c5261-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-s826m\" (UID: \"86efaf93-7f28-4e6a-9add-c8bdd87c5261\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"
Oct 30 19:29:53.515480 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.515108    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86efaf93-7f28-4e6a-9add-c8bdd87c5261-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-s826m\" (UID: \"86efaf93-7f28-4e6a-9add-c8bdd87c5261\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"
Oct 30 19:29:53.515480 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.515318    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86efaf93-7f28-4e6a-9add-c8bdd87c5261-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-s826m\" (UID: \"86efaf93-7f28-4e6a-9add-c8bdd87c5261\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"
Oct 30 19:29:53.515480 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.515475    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86efaf93-7f28-4e6a-9add-c8bdd87c5261-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-s826m\" (UID: \"86efaf93-7f28-4e6a-9add-c8bdd87c5261\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"
Oct 30 19:29:53.515698 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.515644    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86efaf93-7f28-4e6a-9add-c8bdd87c5261-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-s826m\" (UID: \"86efaf93-7f28-4e6a-9add-c8bdd87c5261\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"
Oct 30 19:29:53.517683 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.517659    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86efaf93-7f28-4e6a-9add-c8bdd87c5261-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-s826m\" (UID: \"86efaf93-7f28-4e6a-9add-c8bdd87c5261\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"
Oct 30 19:29:53.520494 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.519255    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86efaf93-7f28-4e6a-9add-c8bdd87c5261-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-s826m\" (UID: \"86efaf93-7f28-4e6a-9add-c8bdd87c5261\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"
Oct 30 19:29:53.544251 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.544200    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrjl5\" (UniqueName: \"kubernetes.io/projected/86efaf93-7f28-4e6a-9add-c8bdd87c5261-kube-api-access-zrjl5\") pod \"maas-default-gateway-openshift-default-5bc884dd87-s826m\" (UID: \"86efaf93-7f28-4e6a-9add-c8bdd87c5261\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"
Oct 30 19:29:53.567912 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.557543    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86efaf93-7f28-4e6a-9add-c8bdd87c5261-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-s826m\" (UID: \"86efaf93-7f28-4e6a-9add-c8bdd87c5261\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"
Oct 30 19:29:53.626873 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.626809    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5f78687d5b-d59pz" event={"ID":"91ee2b9c-4412-415e-817d-d92df100c950","Type":"ContainerStarted","Data":"8c1cbbf26cec5b819801d41c0b92e00d72561a9ba1c056d8d90f1ecf5f50935e"}
Oct 30 19:29:53.647376 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.647343    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"
Oct 30 19:29:53.989066 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.989031    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"]
Oct 30 19:29:53.992440 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:29:53.992407    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86efaf93_7f28_4e6a_9add_c8bdd87c5261.slice/crio-fa15bd9749fd34066eb2fb0c7f20b0fdc9bdb5f7629fa85f3926b4f669b5b8c0 WatchSource:0}: Error finding container fa15bd9749fd34066eb2fb0c7f20b0fdc9bdb5f7629fa85f3926b4f669b5b8c0: Status 404 returned error can't find the container with id fa15bd9749fd34066eb2fb0c7f20b0fdc9bdb5f7629fa85f3926b4f669b5b8c0
Oct 30 19:29:53.996580 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.996533    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:29:53.996700 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.996631    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:29:53.996700 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:53.996677    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:29:54.643786 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:54.643745    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m" event={"ID":"86efaf93-7f28-4e6a-9add-c8bdd87c5261","Type":"ContainerStarted","Data":"141a798d2bf8172e54be80c783484b685b8f6b280a81eee19ff9509e7eee20a7"}
Oct 30 19:29:54.644358 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:54.643819    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m" event={"ID":"86efaf93-7f28-4e6a-9add-c8bdd87c5261","Type":"ContainerStarted","Data":"fa15bd9749fd34066eb2fb0c7f20b0fdc9bdb5f7629fa85f3926b4f669b5b8c0"}
Oct 30 19:29:54.648043 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:54.648018    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"
Oct 30 19:29:54.704042 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:54.703976    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m" podStartSLOduration=1.703956456 podStartE2EDuration="1.703956456s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:29:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:29:54.702450646 +0000 UTC m=+506.545382587" watchObservedRunningTime="2026-10-30 19:29:54.703956456 +0000 UTC m=+506.546888396"
Oct 30 19:29:55.648834 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:55.648798    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-s826m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 30 19:29:55.649276 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:55.648891    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m" podUID="86efaf93-7f28-4e6a-9add-c8bdd87c5261" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 30 19:29:55.650161 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:55.650135    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5f78687d5b-d59pz" event={"ID":"91ee2b9c-4412-415e-817d-d92df100c950","Type":"ContainerStarted","Data":"64535760f2ab8fbee57ae9009825e9230509ccf96e6ef0adcec5e54ba245c125"}
Oct 30 19:29:55.650559 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:55.650538    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5f78687d5b-d59pz"
Oct 30 19:29:55.693969 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:55.693828    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5f78687d5b-d59pz" podStartSLOduration=1.016613144 podStartE2EDuration="3.693814677s" totalImagesPullingTime="2.677201533s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:29:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:29:55.693410929 +0000 UTC m=+507.536342930" watchObservedRunningTime="2026-10-30 19:29:55.693814677 +0000 UTC m=+507.536746615"
Oct 30 19:29:56.648137 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:56.648096    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-s826m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 30 19:29:56.648318 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:56.648155    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m" podUID="86efaf93-7f28-4e6a-9add-c8bdd87c5261" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 30 19:29:57.648404 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:57.648366    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-s826m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 30 19:29:57.648851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:57.648416    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m" podUID="86efaf93-7f28-4e6a-9add-c8bdd87c5261" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 30 19:29:58.647787 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:58.647745    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-s826m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 30 19:29:58.648031 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:58.647819    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m" podUID="86efaf93-7f28-4e6a-9add-c8bdd87c5261" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 30 19:29:59.647766 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:59.647726    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-s826m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 30 19:29:59.648208 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:29:59.647788    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m" podUID="86efaf93-7f28-4e6a-9add-c8bdd87c5261" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 30 19:30:00.146621 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:00.146585    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889810-tk8s2"]
Oct 30 19:30:00.150925 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:00.150908    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-tk8s2"
Oct 30 19:30:00.167944 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:00.167921    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:30:00.183719 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:00.183687    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889810-tk8s2"]
Oct 30 19:30:00.295214 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:00.295186    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rvmz\" (UniqueName: \"kubernetes.io/projected/11d1724f-18d6-45c7-af66-e024459e024a-kube-api-access-6rvmz\") pod \"maas-api-key-cleanup-29889810-tk8s2\" (UID: \"11d1724f-18d6-45c7-af66-e024459e024a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-tk8s2"
Oct 30 19:30:00.295414 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:00.295325    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11d1724f-18d6-45c7-af66-e024459e024a-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889810-tk8s2\" (UID: \"11d1724f-18d6-45c7-af66-e024459e024a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-tk8s2"
Oct 30 19:30:00.396481 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:00.396439    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11d1724f-18d6-45c7-af66-e024459e024a-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889810-tk8s2\" (UID: \"11d1724f-18d6-45c7-af66-e024459e024a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-tk8s2"
Oct 30 19:30:00.396987 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:00.396906    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rvmz\" (UniqueName: \"kubernetes.io/projected/11d1724f-18d6-45c7-af66-e024459e024a-kube-api-access-6rvmz\") pod \"maas-api-key-cleanup-29889810-tk8s2\" (UID: \"11d1724f-18d6-45c7-af66-e024459e024a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-tk8s2"
Oct 30 19:30:00.397333 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:00.397308    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11d1724f-18d6-45c7-af66-e024459e024a-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889810-tk8s2\" (UID: \"11d1724f-18d6-45c7-af66-e024459e024a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-tk8s2"
Oct 30 19:30:00.425584 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:00.425558    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rvmz\" (UniqueName: \"kubernetes.io/projected/11d1724f-18d6-45c7-af66-e024459e024a-kube-api-access-6rvmz\") pod \"maas-api-key-cleanup-29889810-tk8s2\" (UID: \"11d1724f-18d6-45c7-af66-e024459e024a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-tk8s2"
Oct 30 19:30:00.463266 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:00.463230    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-tk8s2"
Oct 30 19:30:00.647780 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:00.647699    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-s826m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 30 19:30:00.647780 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:00.647766    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m" podUID="86efaf93-7f28-4e6a-9add-c8bdd87c5261" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 30 19:30:00.765496 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:00.765464    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889810-tk8s2"]
Oct 30 19:30:00.768911 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:30:00.768878    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11d1724f_18d6_45c7_af66_e024459e024a.slice/crio-68601ca04633732829403d9c21dca12a4247460fd306833c7e46abd5cc54f650 WatchSource:0}: Error finding container 68601ca04633732829403d9c21dca12a4247460fd306833c7e46abd5cc54f650: Status 404 returned error can't find the container with id 68601ca04633732829403d9c21dca12a4247460fd306833c7e46abd5cc54f650
Oct 30 19:30:01.681823 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:01.681736    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-tk8s2" event={"ID":"11d1724f-18d6-45c7-af66-e024459e024a","Type":"ContainerStarted","Data":"68601ca04633732829403d9c21dca12a4247460fd306833c7e46abd5cc54f650"}
Oct 30 19:30:02.607341 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:02.607305    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"
Oct 30 19:30:02.608207 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:02.608104    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"
Oct 30 19:30:02.609382 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:02.609361    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m"
Oct 30 19:30:03.692951 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:03.692845    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-tk8s2" event={"ID":"11d1724f-18d6-45c7-af66-e024459e024a","Type":"ContainerStarted","Data":"9072398295aea59c50edfb3a51e42ec6026752a98228f70ee122ec824a0bb73d"}
Oct 30 19:30:03.742839 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:03.742783    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-tk8s2" podStartSLOduration=1.396905707 podStartE2EDuration="3.742768604s" totalImagesPullingTime="2.345862897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:03.741926625 +0000 UTC m=+515.584858566" watchObservedRunningTime="2026-10-30 19:30:03.742768604 +0000 UTC m=+515.585700521"
Oct 30 19:30:04.699251 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:04.699210    5907 generic.go:356] "Generic (PLEG): container finished" podID="11d1724f-18d6-45c7-af66-e024459e024a" containerID="9072398295aea59c50edfb3a51e42ec6026752a98228f70ee122ec824a0bb73d" exitCode=60
Oct 30 19:30:04.699628 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:04.699278    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-tk8s2" event={"ID":"11d1724f-18d6-45c7-af66-e024459e024a","Type":"ContainerDied","Data":"9072398295aea59c50edfb3a51e42ec6026752a98228f70ee122ec824a0bb73d"}
Oct 30 19:30:04.699628 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:04.699553    5907 scope.go:122] "RemoveContainer" containerID="9072398295aea59c50edfb3a51e42ec6026752a98228f70ee122ec824a0bb73d"
Oct 30 19:30:05.705449 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:05.705351    5907 generic.go:356] "Generic (PLEG): container finished" podID="11d1724f-18d6-45c7-af66-e024459e024a" containerID="39b2de786f60dfacd8573180eeb0094eca9c4fa4e82a442f8aab627956ef79f1" exitCode=60
Oct 30 19:30:05.705898 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:05.705439    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-tk8s2" event={"ID":"11d1724f-18d6-45c7-af66-e024459e024a","Type":"ContainerDied","Data":"39b2de786f60dfacd8573180eeb0094eca9c4fa4e82a442f8aab627956ef79f1"}
Oct 30 19:30:05.705898 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:05.705503    5907 scope.go:122] "RemoveContainer" containerID="9072398295aea59c50edfb3a51e42ec6026752a98228f70ee122ec824a0bb73d"
Oct 30 19:30:05.705898 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:05.705722    5907 scope.go:122] "RemoveContainer" containerID="39b2de786f60dfacd8573180eeb0094eca9c4fa4e82a442f8aab627956ef79f1"
Oct 30 19:30:05.706012 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:30:05.705993    5907 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889810-tk8s2_odh-ai-gateway-infra(11d1724f-18d6-45c7-af66-e024459e024a)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-tk8s2" podUID="11d1724f-18d6-45c7-af66-e024459e024a"
Oct 30 19:30:06.658285 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:06.658257    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5f78687d5b-d59pz"
Oct 30 19:30:06.711626 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:06.711599    5907 scope.go:122] "RemoveContainer" containerID="39b2de786f60dfacd8573180eeb0094eca9c4fa4e82a442f8aab627956ef79f1"
Oct 30 19:30:06.712012 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:30:06.711833    5907 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889810-tk8s2_odh-ai-gateway-infra(11d1724f-18d6-45c7-af66-e024459e024a)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-tk8s2" podUID="11d1724f-18d6-45c7-af66-e024459e024a"
Oct 30 19:30:08.367335 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:08.367292    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"]
Oct 30 19:30:08.372531 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:08.372508    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"
Oct 30 19:30:08.405691 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:08.405656    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"]
Oct 30 19:30:08.482332 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:08.482294    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffa8893a-530b-4232-8e6e-dbe34b30ccda-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-c2sv6\" (UID: \"ffa8893a-530b-4232-8e6e-dbe34b30ccda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"
Oct 30 19:30:08.482509 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:08.482344    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffa8893a-530b-4232-8e6e-dbe34b30ccda-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-c2sv6\" (UID: \"ffa8893a-530b-4232-8e6e-dbe34b30ccda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"
Oct 30 19:30:08.482509 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:08.482368    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffa8893a-530b-4232-8e6e-dbe34b30ccda-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-c2sv6\" (UID: \"ffa8893a-530b-4232-8e6e-dbe34b30ccda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"
Oct 30 19:30:08.482597 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:08.482512    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffa8893a-530b-4232-8e6e-dbe34b30ccda-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-c2sv6\" (UID: \"ffa8893a-530b-4232-8e6e-dbe34b30ccda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"
Oct 30 19:30:08.482597 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:08.482563    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffa8893a-530b-4232-8e6e-dbe34b30ccda-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-c2sv6\" (UID: \"ffa8893a-530b-4232-8e6e-dbe34b30ccda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"
Oct 30 19:30:08.482666 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:08.482627    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62h6f\" (UniqueName: \"kubernetes.io/projected/ffa8893a-530b-4232-8e6e-dbe34b30ccda-kube-api-access-62h6f\") pod \"maas-default-gateway-openshift-default-5bc884dd87-c2sv6\" (UID: \"ffa8893a-530b-4232-8e6e-dbe34b30ccda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"
Oct 30 19:30:08.482722 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:08.482708    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffa8893a-530b-4232-8e6e-dbe34b30ccda-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-c2sv6\" (UID: \"ffa8893a-530b-4232-8e6e-dbe34b30ccda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"
Oct 30 19:30:08.482768 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:08.482743    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffa8893a-530b-4232-8e6e-dbe34b30ccda-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-c2sv6\" (UID: \"ffa8893a-530b-4232-8e6e-dbe34b30ccda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"
Oct 30 19:30:08.482768 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:08.482764    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffa8893a-530b-4232-8e6e-dbe34b30ccda-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-c2sv6\" (UID: \"ffa8893a-530b-4232-8e6e-dbe34b30ccda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"
Oct 30 19:30:08.583460 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:08.583417    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffa8893a-530b-4232-8e6e-dbe34b30ccda-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-c2sv6\" (UID: \"ffa8893a-530b-4232-8e6e-dbe34b30ccda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"
Oct 30 19:30:08.583629 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:08.583481    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffa8893a-530b-4232-8e6e-dbe34b30ccda-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-c2sv6\" (UID: \"ffa8893a-530b-4232-8e6e-dbe34b30ccda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"
Oct 30 19:30:08.583629 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:08.583511    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffa8893a-530b-4232-8e6e-dbe34b30ccda-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-c2sv6\" (UID: \"ffa8893a-530b-4232-8e6e-dbe34b30ccda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"
Oct 30 19:30:08.583629 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:08.583535    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62h6f\" (UniqueName: \"kubernetes.io/projected/ffa8893a-530b-4232-8e6e-dbe34b30ccda-kube-api-access-62h6f\") pod \"maas-default-gateway-openshift-default-5bc884dd87-c2sv6\" (UID: \"ffa8893a-530b-4232-8e6e-dbe34b30ccda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"
Oct 30 19:30:08.583629 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:08.583602    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffa8893a-530b-4232-8e6e-dbe34b30ccda-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-c2sv6\" (UID: \"ffa8893a-530b-4232-8e6e-dbe34b30ccda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"
Oct 30 19:30:08.583963 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:08.583646    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffa8893a-530b-4232-8e6e-dbe34b30ccda-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-c2sv6\" (UID: \"ffa8893a-530b-4232-8e6e-dbe34b30ccda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"
Oct 30 19:30:08.583963 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:08.583675    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffa8893a-530b-4232-8e6e-dbe34b30ccda-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-c2sv6\" (UID: \"ffa8893a-530b-4232-8e6e-dbe34b30ccda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"
Oct 30 19:30:08.583963 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:08.583777    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffa8893a-530b-4232-8e6e-dbe34b30ccda-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-c2sv6\" (UID: \"ffa8893a-530b-4232-8e6e-dbe34b30ccda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"
Oct 30 19:30:08.583963 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:08.583825    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffa8893a-530b-4232-8e6e-dbe34b30ccda-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-c2sv6\" (UID: \"ffa8893a-530b-4232-8e6e-dbe34b30ccda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"
Oct 30 19:30:08.584129 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:08.583981    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffa8893a-530b-4232-8e6e-dbe34b30ccda-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-c2sv6\" (UID: \"ffa8893a-530b-4232-8e6e-dbe34b30ccda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"
Oct 30 19:30:08.584129 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:08.584106    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffa8893a-530b-4232-8e6e-dbe34b30ccda-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-c2sv6\" (UID: \"ffa8893a-530b-4232-8e6e-dbe34b30ccda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"
Oct 30 19:30:08.584398 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:08.584376    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffa8893a-530b-4232-8e6e-dbe34b30ccda-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-c2sv6\" (UID: \"ffa8893a-530b-4232-8e6e-dbe34b30ccda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"
Oct 30 19:30:08.584660 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:08.584639    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffa8893a-530b-4232-8e6e-dbe34b30ccda-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-c2sv6\" (UID: \"ffa8893a-530b-4232-8e6e-dbe34b30ccda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"
Oct 30 19:30:08.584737 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:08.584665    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffa8893a-530b-4232-8e6e-dbe34b30ccda-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-c2sv6\" (UID: \"ffa8893a-530b-4232-8e6e-dbe34b30ccda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"
Oct 30 19:30:08.586551 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:08.586527    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffa8893a-530b-4232-8e6e-dbe34b30ccda-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-c2sv6\" (UID: \"ffa8893a-530b-4232-8e6e-dbe34b30ccda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"
Oct 30 19:30:08.586551 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:08.586540    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffa8893a-530b-4232-8e6e-dbe34b30ccda-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-c2sv6\" (UID: \"ffa8893a-530b-4232-8e6e-dbe34b30ccda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"
Oct 30 19:30:08.604793 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:08.604767    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62h6f\" (UniqueName: \"kubernetes.io/projected/ffa8893a-530b-4232-8e6e-dbe34b30ccda-kube-api-access-62h6f\") pod \"maas-default-gateway-openshift-default-5bc884dd87-c2sv6\" (UID: \"ffa8893a-530b-4232-8e6e-dbe34b30ccda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"
Oct 30 19:30:08.604920 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:08.604795    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffa8893a-530b-4232-8e6e-dbe34b30ccda-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-c2sv6\" (UID: \"ffa8893a-530b-4232-8e6e-dbe34b30ccda\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"
Oct 30 19:30:08.689238 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:08.689144    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"
Oct 30 19:30:09.008807 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:09.008779    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"]
Oct 30 19:30:09.011543 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:30:09.011517    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffa8893a_530b_4232_8e6e_dbe34b30ccda.slice/crio-20ee46298aa92c72e1940aea6f8b6ab775afb77ecd08b567ff7830da87845e10 WatchSource:0}: Error finding container 20ee46298aa92c72e1940aea6f8b6ab775afb77ecd08b567ff7830da87845e10: Status 404 returned error can't find the container with id 20ee46298aa92c72e1940aea6f8b6ab775afb77ecd08b567ff7830da87845e10
Oct 30 19:30:09.013709 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:09.013673    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:30:09.013791 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:09.013739    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:30:09.013791 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:09.013769    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:30:09.740893 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:09.740834    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6" event={"ID":"ffa8893a-530b-4232-8e6e-dbe34b30ccda","Type":"ContainerStarted","Data":"5d794a6f97517e8fe74d35ffa0af97d686d89879004f8704fab8a720d947416d"}
Oct 30 19:30:09.741321 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:09.740946    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6" event={"ID":"ffa8893a-530b-4232-8e6e-dbe34b30ccda","Type":"ContainerStarted","Data":"20ee46298aa92c72e1940aea6f8b6ab775afb77ecd08b567ff7830da87845e10"}
Oct 30 19:30:09.789139 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:09.789089    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6" podStartSLOduration=1.789074262 podStartE2EDuration="1.789074262s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:09.787027664 +0000 UTC m=+521.629959602" watchObservedRunningTime="2026-10-30 19:30:09.789074262 +0000 UTC m=+521.632006200"
Oct 30 19:30:10.690473 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:10.690370    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"
Oct 30 19:30:10.692036 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:10.692008    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-c2sv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Oct 30 19:30:10.692102 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:10.692066    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6" podUID="ffa8893a-530b-4232-8e6e-dbe34b30ccda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Oct 30 19:30:11.689953 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:11.689915    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-c2sv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Oct 30 19:30:11.690362 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:11.689986    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6" podUID="ffa8893a-530b-4232-8e6e-dbe34b30ccda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Oct 30 19:30:12.689794 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:12.689755    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-c2sv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Oct 30 19:30:12.689988 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:12.689828    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6" podUID="ffa8893a-530b-4232-8e6e-dbe34b30ccda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Oct 30 19:30:13.689757 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:13.689714    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-c2sv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Oct 30 19:30:13.689980 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:13.689776    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6" podUID="ffa8893a-530b-4232-8e6e-dbe34b30ccda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Oct 30 19:30:14.689905 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:14.689845    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-c2sv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Oct 30 19:30:14.690316 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:14.689961    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6" podUID="ffa8893a-530b-4232-8e6e-dbe34b30ccda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Oct 30 19:30:15.689988 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:15.689955    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-c2sv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Oct 30 19:30:15.690387 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:15.690007    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6" podUID="ffa8893a-530b-4232-8e6e-dbe34b30ccda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Oct 30 19:30:17.280930 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:17.280896    5907 scope.go:122] "RemoveContainer" containerID="39b2de786f60dfacd8573180eeb0094eca9c4fa4e82a442f8aab627956ef79f1"
Oct 30 19:30:17.283385 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:17.283362    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"
Oct 30 19:30:17.284108 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:17.284086    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"
Oct 30 19:30:17.284912 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:17.284891    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6"
Oct 30 19:30:17.501356 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:30:17.501220    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:30:17.783843 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:17.783805    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-tk8s2" event={"ID":"11d1724f-18d6-45c7-af66-e024459e024a","Type":"ContainerStarted","Data":"3cd8cd421e1a4925c8a26afd042876b96e1760932ecd2ed5dfc0c6eb49e7fb8e"}
Oct 30 19:30:18.790712 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:18.790676    5907 generic.go:356] "Generic (PLEG): container finished" podID="11d1724f-18d6-45c7-af66-e024459e024a" containerID="3cd8cd421e1a4925c8a26afd042876b96e1760932ecd2ed5dfc0c6eb49e7fb8e" exitCode=60
Oct 30 19:30:18.791158 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:18.790753    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-tk8s2" event={"ID":"11d1724f-18d6-45c7-af66-e024459e024a","Type":"ContainerDied","Data":"3cd8cd421e1a4925c8a26afd042876b96e1760932ecd2ed5dfc0c6eb49e7fb8e"}
Oct 30 19:30:18.791158 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:18.790820    5907 scope.go:122] "RemoveContainer" containerID="39b2de786f60dfacd8573180eeb0094eca9c4fa4e82a442f8aab627956ef79f1"
Oct 30 19:30:18.791288 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:18.791267    5907 scope.go:122] "RemoveContainer" containerID="3cd8cd421e1a4925c8a26afd042876b96e1760932ecd2ed5dfc0c6eb49e7fb8e"
Oct 30 19:30:18.791545 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:30:18.791526    5907 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29889810-tk8s2_odh-ai-gateway-infra(11d1724f-18d6-45c7-af66-e024459e024a)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-tk8s2" podUID="11d1724f-18d6-45c7-af66-e024459e024a"
Oct 30 19:30:18.847757 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:18.847724    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889810-tk8s2"]
Oct 30 19:30:19.928261 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:19.928232    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-tk8s2"
Oct 30 19:30:20.022440 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:20.022411    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/11d1724f-18d6-45c7-af66-e024459e024a-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11d1724f-18d6-45c7-af66-e024459e024a-maas-api-service-ca\") pod \"11d1724f-18d6-45c7-af66-e024459e024a\" (UID: \"11d1724f-18d6-45c7-af66-e024459e024a\") "
Oct 30 19:30:20.022630 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:20.022544    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/11d1724f-18d6-45c7-af66-e024459e024a-kube-api-access-6rvmz\" (UniqueName: \"kubernetes.io/projected/11d1724f-18d6-45c7-af66-e024459e024a-kube-api-access-6rvmz\") pod \"11d1724f-18d6-45c7-af66-e024459e024a\" (UID: \"11d1724f-18d6-45c7-af66-e024459e024a\") "
Oct 30 19:30:20.022836 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:20.022807    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11d1724f-18d6-45c7-af66-e024459e024a-maas-api-service-ca" pod "11d1724f-18d6-45c7-af66-e024459e024a" (UID: "11d1724f-18d6-45c7-af66-e024459e024a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:20.024749 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:20.024728    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11d1724f-18d6-45c7-af66-e024459e024a-kube-api-access-6rvmz" pod "11d1724f-18d6-45c7-af66-e024459e024a" (UID: "11d1724f-18d6-45c7-af66-e024459e024a"). InnerVolumeSpecName "kube-api-access-6rvmz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:20.123353 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:20.123320    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rvmz\" (UniqueName: \"kubernetes.io/projected/11d1724f-18d6-45c7-af66-e024459e024a-kube-api-access-6rvmz\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:30:20.123353 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:20.123352    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11d1724f-18d6-45c7-af66-e024459e024a-maas-api-service-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:30:20.803437 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:20.803406    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-tk8s2"
Oct 30 19:30:20.803629 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:20.803406    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-tk8s2" event={"ID":"11d1724f-18d6-45c7-af66-e024459e024a","Type":"ContainerDied","Data":"68601ca04633732829403d9c21dca12a4247460fd306833c7e46abd5cc54f650"}
Oct 30 19:30:20.803629 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:20.803578    5907 scope.go:122] "RemoveContainer" containerID="3cd8cd421e1a4925c8a26afd042876b96e1760932ecd2ed5dfc0c6eb49e7fb8e"
Oct 30 19:30:20.860412 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:20.860383    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889810-tk8s2"]
Oct 30 19:30:20.873267 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:20.873238    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889810-tk8s2"]
Oct 30 19:30:22.288337 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:22.288300    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="11d1724f-18d6-45c7-af66-e024459e024a" path="/var/lib/kubelet/pods/11d1724f-18d6-45c7-af66-e024459e024a/volumes"
Oct 30 19:30:44.118991 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.118916    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"]
Oct 30 19:30:44.119394 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.119380    5907 cpu_manager.go:414] "containerMap: removing container" podUID="11d1724f-18d6-45c7-af66-e024459e024a" containerName="cleanup"
Oct 30 19:30:44.119435 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.119396    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="11d1724f-18d6-45c7-af66-e024459e024a" containerName="cleanup"
Oct 30 19:30:44.119435 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.119415    5907 cpu_manager.go:414] "containerMap: removing container" podUID="11d1724f-18d6-45c7-af66-e024459e024a" containerName="cleanup"
Oct 30 19:30:44.119435 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.119420    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="11d1724f-18d6-45c7-af66-e024459e024a" containerName="cleanup"
Oct 30 19:30:44.119528 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.119497    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="11d1724f-18d6-45c7-af66-e024459e024a" containerName="cleanup"
Oct 30 19:30:44.119528 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.119507    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="11d1724f-18d6-45c7-af66-e024459e024a" containerName="cleanup"
Oct 30 19:30:44.119528 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.119515    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="11d1724f-18d6-45c7-af66-e024459e024a" containerName="cleanup"
Oct 30 19:30:44.122645 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.122622    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:30:44.138487 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.138467    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-d797d2-openshift-default-dockercfg-qxcvg\""
Oct 30 19:30:44.154085 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.154054    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"]
Oct 30 19:30:44.267245 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.267208    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b70d17b-10c5-4886-a033-caef763f1406-istio-data\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:30:44.267436 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.267267    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b70d17b-10c5-4886-a033-caef763f1406-istio-envoy\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:30:44.267436 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.267297    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b70d17b-10c5-4886-a033-caef763f1406-istio-podinfo\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:30:44.267550 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.267467    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b70d17b-10c5-4886-a033-caef763f1406-workload-certs\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:30:44.267586 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.267566    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b70d17b-10c5-4886-a033-caef763f1406-istio-token\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:30:44.267629 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.267590    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b70d17b-10c5-4886-a033-caef763f1406-credential-socket\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:30:44.267704 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.267688    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b70d17b-10c5-4886-a033-caef763f1406-istiod-ca-cert\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:30:44.267773 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.267755    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttrln\" (UniqueName: \"kubernetes.io/projected/1b70d17b-10c5-4886-a033-caef763f1406-kube-api-access-ttrln\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:30:44.267841 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.267828    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b70d17b-10c5-4886-a033-caef763f1406-workload-socket\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:30:44.369664 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.369574    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttrln\" (UniqueName: \"kubernetes.io/projected/1b70d17b-10c5-4886-a033-caef763f1406-kube-api-access-ttrln\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:30:44.369664 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.369623    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b70d17b-10c5-4886-a033-caef763f1406-workload-socket\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:30:44.369664 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.369664    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b70d17b-10c5-4886-a033-caef763f1406-istio-data\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:30:44.369998 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.369697    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b70d17b-10c5-4886-a033-caef763f1406-istio-envoy\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:30:44.369998 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.369718    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b70d17b-10c5-4886-a033-caef763f1406-istio-podinfo\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:30:44.369998 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.369753    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b70d17b-10c5-4886-a033-caef763f1406-workload-certs\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:30:44.369998 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.369815    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b70d17b-10c5-4886-a033-caef763f1406-istio-token\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:30:44.369998 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.369839    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b70d17b-10c5-4886-a033-caef763f1406-credential-socket\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:30:44.369998 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.369919    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b70d17b-10c5-4886-a033-caef763f1406-istiod-ca-cert\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:30:44.370339 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.370127    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b70d17b-10c5-4886-a033-caef763f1406-istio-data\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:30:44.370339 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.370288    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b70d17b-10c5-4886-a033-caef763f1406-workload-certs\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:30:44.370339 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.370317    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b70d17b-10c5-4886-a033-caef763f1406-workload-socket\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:30:44.370498 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.370373    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b70d17b-10c5-4886-a033-caef763f1406-credential-socket\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:30:44.370722 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.370608    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b70d17b-10c5-4886-a033-caef763f1406-istiod-ca-cert\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:30:44.372217 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.372192    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b70d17b-10c5-4886-a033-caef763f1406-istio-envoy\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:30:44.372521 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.372502    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b70d17b-10c5-4886-a033-caef763f1406-istio-podinfo\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:30:44.391769 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.391736    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b70d17b-10c5-4886-a033-caef763f1406-istio-token\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:30:44.393014 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.392993    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttrln\" (UniqueName: \"kubernetes.io/projected/1b70d17b-10c5-4886-a033-caef763f1406-kube-api-access-ttrln\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:30:44.412196 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.412159    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"]
Oct 30 19:30:44.412667 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.412650    5907 cpu_manager.go:414] "containerMap: removing container" podUID="11d1724f-18d6-45c7-af66-e024459e024a" containerName="cleanup"
Oct 30 19:30:44.412766 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.412668    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="11d1724f-18d6-45c7-af66-e024459e024a" containerName="cleanup"
Oct 30 19:30:44.417770 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.417739    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:30:44.438015 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.437980    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:30:44.442039 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.442007    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"]
Oct 30 19:30:44.442180 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.442104    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-3735b2-openshift-default-dockercfg-z8mrd\""
Oct 30 19:30:44.571810 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.571775    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/282f4fd8-512c-41b9-9989-0b84f2de1ce5-workload-socket\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:30:44.571989 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.571839    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/282f4fd8-512c-41b9-9989-0b84f2de1ce5-workload-certs\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:30:44.571989 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.571916    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/282f4fd8-512c-41b9-9989-0b84f2de1ce5-istio-data\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:30:44.572066 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.571983    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/282f4fd8-512c-41b9-9989-0b84f2de1ce5-istiod-ca-cert\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:30:44.572066 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.572020    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/282f4fd8-512c-41b9-9989-0b84f2de1ce5-istio-podinfo\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:30:44.572129 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.572061    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvsjd\" (UniqueName: \"kubernetes.io/projected/282f4fd8-512c-41b9-9989-0b84f2de1ce5-kube-api-access-pvsjd\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:30:44.572129 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.572089    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/282f4fd8-512c-41b9-9989-0b84f2de1ce5-istio-token\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:30:44.572129 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.572109    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/282f4fd8-512c-41b9-9989-0b84f2de1ce5-credential-socket\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:30:44.572217 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.572179    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/282f4fd8-512c-41b9-9989-0b84f2de1ce5-istio-envoy\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:30:44.673433 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.673340    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/282f4fd8-512c-41b9-9989-0b84f2de1ce5-istio-envoy\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:30:44.673433 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.673400    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/282f4fd8-512c-41b9-9989-0b84f2de1ce5-workload-socket\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:30:44.673664 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.673443    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/282f4fd8-512c-41b9-9989-0b84f2de1ce5-workload-certs\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:30:44.673664 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.673471    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/282f4fd8-512c-41b9-9989-0b84f2de1ce5-istio-data\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:30:44.673664 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.673498    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/282f4fd8-512c-41b9-9989-0b84f2de1ce5-istiod-ca-cert\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:30:44.673664 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.673527    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/282f4fd8-512c-41b9-9989-0b84f2de1ce5-istio-podinfo\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:30:44.673664 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.673578    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvsjd\" (UniqueName: \"kubernetes.io/projected/282f4fd8-512c-41b9-9989-0b84f2de1ce5-kube-api-access-pvsjd\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:30:44.673664 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.673621    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/282f4fd8-512c-41b9-9989-0b84f2de1ce5-istio-token\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:30:44.673664 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.673649    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/282f4fd8-512c-41b9-9989-0b84f2de1ce5-credential-socket\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:30:44.674044 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.673813    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/282f4fd8-512c-41b9-9989-0b84f2de1ce5-workload-socket\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:30:44.674208 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.674188    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/282f4fd8-512c-41b9-9989-0b84f2de1ce5-istiod-ca-cert\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:30:44.674475 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.674285    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/282f4fd8-512c-41b9-9989-0b84f2de1ce5-workload-certs\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:30:44.674475 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.674299    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/282f4fd8-512c-41b9-9989-0b84f2de1ce5-credential-socket\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:30:44.674475 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.674320    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/282f4fd8-512c-41b9-9989-0b84f2de1ce5-istio-data\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:30:44.675981 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.675952    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/282f4fd8-512c-41b9-9989-0b84f2de1ce5-istio-envoy\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:30:44.676180 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.676164    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/282f4fd8-512c-41b9-9989-0b84f2de1ce5-istio-podinfo\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:30:44.703492 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.703459    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvsjd\" (UniqueName: \"kubernetes.io/projected/282f4fd8-512c-41b9-9989-0b84f2de1ce5-kube-api-access-pvsjd\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:30:44.704008 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.703983    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/282f4fd8-512c-41b9-9989-0b84f2de1ce5-istio-token\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:30:44.733751 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.733207    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:30:44.736685 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.736655    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"]
Oct 30 19:30:44.740779 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.740754    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:30:44.768357 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.768327    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-dd23d1-openshift-default-dockercfg-qvtg4\""
Oct 30 19:30:44.786514 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.786486    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"]
Oct 30 19:30:44.808919 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.808890    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"]
Oct 30 19:30:44.818314 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.815521    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:30:44.818314 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.815589    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:30:44.818314 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.815620    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:30:44.875908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.875874    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-istio-token\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:30:44.876055 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.875942    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm9l5\" (UniqueName: \"kubernetes.io/projected/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-kube-api-access-xm9l5\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:30:44.876055 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.876014    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-istio-podinfo\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:30:44.876055 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.876038    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-istio-envoy\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:30:44.876163 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.876066    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-istio-data\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:30:44.876199 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.876175    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-workload-certs\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:30:44.876230 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.876207    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-istiod-ca-cert\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:30:44.876265 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.876233    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-workload-socket\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:30:44.876302 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.876290    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-credential-socket\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:30:44.936371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.936267    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk" event={"ID":"1b70d17b-10c5-4886-a033-caef763f1406","Type":"ContainerStarted","Data":"c34293f861a664a7de82259386c8ba601b042478b12ea37bf1d692111d7a5349"}
Oct 30 19:30:44.936371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.936327    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk" event={"ID":"1b70d17b-10c5-4886-a033-caef763f1406","Type":"ContainerStarted","Data":"f68109273bf540cf4e07952460b4b3e37a6f18b0fccca5ca72492a0b5b16bea7"}
Oct 30 19:30:44.977841 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.977810    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-istio-podinfo\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:30:44.977841 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.977845    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-istio-envoy\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:30:44.978127 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.977904    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-istio-data\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:30:44.978127 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.977935    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-workload-certs\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:30:44.978127 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.977963    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-istiod-ca-cert\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:30:44.978127 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.977994    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-workload-socket\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:30:44.978127 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.978034    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-credential-socket\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:30:44.978386 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.978179    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-istio-token\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:30:44.978386 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.978210    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm9l5\" (UniqueName: \"kubernetes.io/projected/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-kube-api-access-xm9l5\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:30:44.979290 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.978834    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-workload-socket\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:30:44.979290 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.978892    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-credential-socket\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:30:44.979290 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.979072    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-istio-data\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:30:44.979290 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.979101    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-workload-certs\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:30:44.979290 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.979225    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-istiod-ca-cert\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:30:44.986112 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.986029    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-istio-envoy\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:30:44.986903 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:44.986773    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-istio-podinfo\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:30:45.010310 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:45.010257    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm9l5\" (UniqueName: \"kubernetes.io/projected/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-kube-api-access-xm9l5\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:30:45.010729 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:45.010704    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-istio-token\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:30:45.070695 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:45.070662    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:30:45.115677 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:45.115054    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk" podStartSLOduration=1.115030939 podStartE2EDuration="1.115030939s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:45.031573002 +0000 UTC m=+556.874504941" watchObservedRunningTime="2026-10-30 19:30:45.115030939 +0000 UTC m=+556.957962880"
Oct 30 19:30:45.117048 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:45.117023    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"]
Oct 30 19:30:45.120949 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:30:45.120845    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod282f4fd8_512c_41b9_9989_0b84f2de1ce5.slice/crio-36c3c91ce9e5ddd221b84e493f009ed8c9763ee444296777ef032292142d0cc3 WatchSource:0}: Error finding container 36c3c91ce9e5ddd221b84e493f009ed8c9763ee444296777ef032292142d0cc3: Status 404 returned error can't find the container with id 36c3c91ce9e5ddd221b84e493f009ed8c9763ee444296777ef032292142d0cc3
Oct 30 19:30:45.123734 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:45.123696    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:30:45.123848 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:45.123772    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:30:45.123848 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:45.123805    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:30:45.428019 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:45.427732    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"]
Oct 30 19:30:45.440318 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:45.439959    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:30:45.442466 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:45.442382    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:30:45.442568 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:45.442487    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:30:45.442568 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:45.442532    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:30:45.446758 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:45.446271    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:30:45.943237 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:45.943201    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc" event={"ID":"282f4fd8-512c-41b9-9989-0b84f2de1ce5","Type":"ContainerStarted","Data":"01633586736e0aaf824342a7876015a1a00880bfcafb68c4caa6f9f362652fd1"}
Oct 30 19:30:45.943433 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:45.943258    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc" event={"ID":"282f4fd8-512c-41b9-9989-0b84f2de1ce5","Type":"ContainerStarted","Data":"36c3c91ce9e5ddd221b84e493f009ed8c9763ee444296777ef032292142d0cc3"}
Oct 30 19:30:45.944913 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:45.944886    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl" event={"ID":"ca6dddc7-2d29-4500-ab3f-7c982fc8064d","Type":"ContainerStarted","Data":"e0b5013aeda8e3120b40f658b4ca5bbce6b6b3cd404cf13c3c06bdd26c6d88e7"}
Oct 30 19:30:45.945021 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:45.944934    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl" event={"ID":"ca6dddc7-2d29-4500-ab3f-7c982fc8064d","Type":"ContainerStarted","Data":"097f0af3bec906dfec2d327258313a3ec3a3e5277fdb8a1e41e0873ff208562e"}
Oct 30 19:30:45.945111 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:45.945094    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:30:45.946121 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:45.946102    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:30:45.992069 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:45.991984    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc" podStartSLOduration=1.991968465 podStartE2EDuration="1.991968465s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:45.990300829 +0000 UTC m=+557.833232768" watchObservedRunningTime="2026-10-30 19:30:45.991968465 +0000 UTC m=+557.834900403"
Oct 30 19:30:46.071237 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:46.071196    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:30:46.076409 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:46.076376    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:30:46.086943 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:46.086831    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl" podStartSLOduration=2.086816638 podStartE2EDuration="2.086816638s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:46.084875092 +0000 UTC m=+557.927807034" watchObservedRunningTime="2026-10-30 19:30:46.086816638 +0000 UTC m=+557.929748577"
Oct 30 19:30:46.734435 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:46.734396    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:30:46.739636 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:46.739611    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:30:46.950266 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:46.950221    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:30:46.950266 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:46.950274    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:30:46.950917 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:46.950894    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:30:46.951419 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:46.951402    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:30:47.499619 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:30:47.499578    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:30:50.227915 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:50.227854    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-d797d2-67d78678ff-t4ccl"]
Oct 30 19:30:50.231934 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:50.231912    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-d797d2-67d78678ff-t4ccl"
Oct 30 19:30:50.240991 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:50.240961    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgh9j\" (UniqueName: \"kubernetes.io/projected/55e11ef7-bf87-4002-93e8-46af5da4c8f0-kube-api-access-fgh9j\") pod \"payload-pre-processing-e2e-worker-w1-d797d2-67d78678ff-t4ccl\" (UID: \"55e11ef7-bf87-4002-93e8-46af5da4c8f0\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-d797d2-67d78678ff-t4ccl"
Oct 30 19:30:50.241122 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:50.241043    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55e11ef7-bf87-4002-93e8-46af5da4c8f0-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-d797d2-67d78678ff-t4ccl\" (UID: \"55e11ef7-bf87-4002-93e8-46af5da4c8f0\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-d797d2-67d78678ff-t4ccl"
Oct 30 19:30:50.251549 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:50.251511    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-d797d2\""
Oct 30 19:30:50.263928 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:50.263895    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-d797d2-67d78678ff-t4ccl"]
Oct 30 19:30:50.341964 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:50.341920    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgh9j\" (UniqueName: \"kubernetes.io/projected/55e11ef7-bf87-4002-93e8-46af5da4c8f0-kube-api-access-fgh9j\") pod \"payload-pre-processing-e2e-worker-w1-d797d2-67d78678ff-t4ccl\" (UID: \"55e11ef7-bf87-4002-93e8-46af5da4c8f0\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-d797d2-67d78678ff-t4ccl"
Oct 30 19:30:50.342185 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:50.342019    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55e11ef7-bf87-4002-93e8-46af5da4c8f0-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-d797d2-67d78678ff-t4ccl\" (UID: \"55e11ef7-bf87-4002-93e8-46af5da4c8f0\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-d797d2-67d78678ff-t4ccl"
Oct 30 19:30:50.342771 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:50.342749    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55e11ef7-bf87-4002-93e8-46af5da4c8f0-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-d797d2-67d78678ff-t4ccl\" (UID: \"55e11ef7-bf87-4002-93e8-46af5da4c8f0\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-d797d2-67d78678ff-t4ccl"
Oct 30 19:30:50.367531 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:50.367496    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgh9j\" (UniqueName: \"kubernetes.io/projected/55e11ef7-bf87-4002-93e8-46af5da4c8f0-kube-api-access-fgh9j\") pod \"payload-pre-processing-e2e-worker-w1-d797d2-67d78678ff-t4ccl\" (UID: \"55e11ef7-bf87-4002-93e8-46af5da4c8f0\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-d797d2-67d78678ff-t4ccl"
Oct 30 19:30:50.548163 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:50.548072    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-d797d2-67d78678ff-t4ccl"
Oct 30 19:30:50.858356 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:50.858331    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-d797d2-67d78678ff-t4ccl"]
Oct 30 19:30:50.860929 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:30:50.860901    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55e11ef7_bf87_4002_93e8_46af5da4c8f0.slice/crio-5f4ee057a5726af04e5c1b1bd083b52fcbdbc449b7e89281e0b51596b79ce4c1 WatchSource:0}: Error finding container 5f4ee057a5726af04e5c1b1bd083b52fcbdbc449b7e89281e0b51596b79ce4c1: Status 404 returned error can't find the container with id 5f4ee057a5726af04e5c1b1bd083b52fcbdbc449b7e89281e0b51596b79ce4c1
Oct 30 19:30:50.971678 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:50.971647    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-d797d2-67d78678ff-t4ccl" event={"ID":"55e11ef7-bf87-4002-93e8-46af5da4c8f0","Type":"ContainerStarted","Data":"39096ffc723273012185e392683bfad7f6cd989318acbe01c9cb6377615948b9"}
Oct 30 19:30:50.971847 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:50.971703    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-d797d2-67d78678ff-t4ccl" event={"ID":"55e11ef7-bf87-4002-93e8-46af5da4c8f0","Type":"ContainerStarted","Data":"5f4ee057a5726af04e5c1b1bd083b52fcbdbc449b7e89281e0b51596b79ce4c1"}
Oct 30 19:30:50.971847 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:50.971787    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-d797d2-67d78678ff-t4ccl"
Oct 30 19:30:51.033043 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:51.027334    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-d797d2-67d78678ff-t4ccl" podStartSLOduration=1.027291038 podStartE2EDuration="1.027291038s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:51.019574999 +0000 UTC m=+562.862506939" watchObservedRunningTime="2026-10-30 19:30:51.027291038 +0000 UTC m=+562.870222978"
Oct 30 19:30:51.848757 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:51.848719    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-ef9a5a-59fc6bf976-zfzks"]
Oct 30 19:30:51.852856 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:51.852832    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-ef9a5a-59fc6bf976-zfzks"
Oct 30 19:30:51.860995 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:51.860854    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09920b13-483e-47d3-89f0-c22163568979-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-ef9a5a-59fc6bf976-zfzks\" (UID: \"09920b13-483e-47d3-89f0-c22163568979\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-ef9a5a-59fc6bf976-zfzks"
Oct 30 19:30:51.861116 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:51.861070    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzc2b\" (UniqueName: \"kubernetes.io/projected/09920b13-483e-47d3-89f0-c22163568979-kube-api-access-jzc2b\") pod \"payload-processing-e2e-worker-w6-ef9a5a-59fc6bf976-zfzks\" (UID: \"09920b13-483e-47d3-89f0-c22163568979\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-ef9a5a-59fc6bf976-zfzks"
Oct 30 19:30:51.862292 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:51.862268    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-ef9a5a-59fc6bf976-zfzks"]
Oct 30 19:30:51.876670 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:51.876625    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w6-ef9a5a\""
Oct 30 19:30:51.876670 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:51.876635    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-ef9a5a-796b5cf9dc-kmcg7"]
Oct 30 19:30:51.881204 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:51.881182    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ef9a5a-796b5cf9dc-kmcg7"
Oct 30 19:30:51.901994 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:51.901968    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-ef9a5a-796b5cf9dc-kmcg7"]
Oct 30 19:30:51.903320 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:51.903294    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w6-ef9a5a-dockercfg-xbnt8\""
Oct 30 19:30:51.985884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:51.973700    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzc2b\" (UniqueName: \"kubernetes.io/projected/09920b13-483e-47d3-89f0-c22163568979-kube-api-access-jzc2b\") pod \"payload-processing-e2e-worker-w6-ef9a5a-59fc6bf976-zfzks\" (UID: \"09920b13-483e-47d3-89f0-c22163568979\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-ef9a5a-59fc6bf976-zfzks"
Oct 30 19:30:51.985884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:51.973815    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnqk2\" (UniqueName: \"kubernetes.io/projected/3b797e1f-bd98-4740-b15d-6ee4e1029d2a-kube-api-access-rnqk2\") pod \"payload-pre-processing-e2e-worker-w6-ef9a5a-796b5cf9dc-kmcg7\" (UID: \"3b797e1f-bd98-4740-b15d-6ee4e1029d2a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ef9a5a-796b5cf9dc-kmcg7"
Oct 30 19:30:51.985884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:51.973883    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09920b13-483e-47d3-89f0-c22163568979-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-ef9a5a-59fc6bf976-zfzks\" (UID: \"09920b13-483e-47d3-89f0-c22163568979\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-ef9a5a-59fc6bf976-zfzks"
Oct 30 19:30:51.985884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:51.973953    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b797e1f-bd98-4740-b15d-6ee4e1029d2a-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-ef9a5a-796b5cf9dc-kmcg7\" (UID: \"3b797e1f-bd98-4740-b15d-6ee4e1029d2a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ef9a5a-796b5cf9dc-kmcg7"
Oct 30 19:30:51.985884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:51.975197    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09920b13-483e-47d3-89f0-c22163568979-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-ef9a5a-59fc6bf976-zfzks\" (UID: \"09920b13-483e-47d3-89f0-c22163568979\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-ef9a5a-59fc6bf976-zfzks"
Oct 30 19:30:52.022626 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:52.022593    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzc2b\" (UniqueName: \"kubernetes.io/projected/09920b13-483e-47d3-89f0-c22163568979-kube-api-access-jzc2b\") pod \"payload-processing-e2e-worker-w6-ef9a5a-59fc6bf976-zfzks\" (UID: \"09920b13-483e-47d3-89f0-c22163568979\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-ef9a5a-59fc6bf976-zfzks"
Oct 30 19:30:52.075119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:52.075082    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnqk2\" (UniqueName: \"kubernetes.io/projected/3b797e1f-bd98-4740-b15d-6ee4e1029d2a-kube-api-access-rnqk2\") pod \"payload-pre-processing-e2e-worker-w6-ef9a5a-796b5cf9dc-kmcg7\" (UID: \"3b797e1f-bd98-4740-b15d-6ee4e1029d2a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ef9a5a-796b5cf9dc-kmcg7"
Oct 30 19:30:52.075314 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:52.075174    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b797e1f-bd98-4740-b15d-6ee4e1029d2a-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-ef9a5a-796b5cf9dc-kmcg7\" (UID: \"3b797e1f-bd98-4740-b15d-6ee4e1029d2a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ef9a5a-796b5cf9dc-kmcg7"
Oct 30 19:30:52.076020 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:52.075997    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b797e1f-bd98-4740-b15d-6ee4e1029d2a-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-ef9a5a-796b5cf9dc-kmcg7\" (UID: \"3b797e1f-bd98-4740-b15d-6ee4e1029d2a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ef9a5a-796b5cf9dc-kmcg7"
Oct 30 19:30:52.101059 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:52.101001    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnqk2\" (UniqueName: \"kubernetes.io/projected/3b797e1f-bd98-4740-b15d-6ee4e1029d2a-kube-api-access-rnqk2\") pod \"payload-pre-processing-e2e-worker-w6-ef9a5a-796b5cf9dc-kmcg7\" (UID: \"3b797e1f-bd98-4740-b15d-6ee4e1029d2a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ef9a5a-796b5cf9dc-kmcg7"
Oct 30 19:30:52.169565 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:52.169531    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-ef9a5a-59fc6bf976-zfzks"
Oct 30 19:30:52.196024 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:52.195995    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ef9a5a-796b5cf9dc-kmcg7"
Oct 30 19:30:52.472591 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:52.472540    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-ef9a5a-59fc6bf976-zfzks"]
Oct 30 19:30:52.475177 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:30:52.475131    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09920b13_483e_47d3_89f0_c22163568979.slice/crio-4f9e9fc15c8a115c6748a53ff2a5dab38eb3240ab074fd016b9f1eee9d3a863d WatchSource:0}: Error finding container 4f9e9fc15c8a115c6748a53ff2a5dab38eb3240ab074fd016b9f1eee9d3a863d: Status 404 returned error can't find the container with id 4f9e9fc15c8a115c6748a53ff2a5dab38eb3240ab074fd016b9f1eee9d3a863d
Oct 30 19:30:52.509518 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:52.509492    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-ef9a5a-796b5cf9dc-kmcg7"]
Oct 30 19:30:52.706802 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:52.706768    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-544fc8-cb5b6cbd7-h7tjd"]
Oct 30 19:30:52.710801 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:52.710778    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-544fc8-cb5b6cbd7-h7tjd"
Oct 30 19:30:52.730207 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:52.730145    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w5-544fc8\""
Oct 30 19:30:52.741173 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:52.741148    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-544fc8-cb5b6cbd7-h7tjd"]
Oct 30 19:30:52.744833 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:52.744804    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w5-544fc8-dockercfg-prsqb\""
Oct 30 19:30:52.785602 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:52.785564    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97367526-d21f-45f0-bc3f-d36ae4a1fe7c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-544fc8-cb5b6cbd7-h7tjd\" (UID: \"97367526-d21f-45f0-bc3f-d36ae4a1fe7c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-544fc8-cb5b6cbd7-h7tjd"
Oct 30 19:30:52.785774 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:52.785726    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vz59m\" (UniqueName: \"kubernetes.io/projected/97367526-d21f-45f0-bc3f-d36ae4a1fe7c-kube-api-access-vz59m\") pod \"payload-pre-processing-e2e-worker-w5-544fc8-cb5b6cbd7-h7tjd\" (UID: \"97367526-d21f-45f0-bc3f-d36ae4a1fe7c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-544fc8-cb5b6cbd7-h7tjd"
Oct 30 19:30:52.798846 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:52.798811    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-544fc8-78bc5bc8bd-wjdrd"]
Oct 30 19:30:52.802627 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:52.802607    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-544fc8-78bc5bc8bd-wjdrd"
Oct 30 19:30:52.828741 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:52.828702    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-544fc8-78bc5bc8bd-wjdrd"]
Oct 30 19:30:52.889731 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:52.889671    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97367526-d21f-45f0-bc3f-d36ae4a1fe7c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-544fc8-cb5b6cbd7-h7tjd\" (UID: \"97367526-d21f-45f0-bc3f-d36ae4a1fe7c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-544fc8-cb5b6cbd7-h7tjd"
Oct 30 19:30:52.894843 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:52.894802    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz59m\" (UniqueName: \"kubernetes.io/projected/97367526-d21f-45f0-bc3f-d36ae4a1fe7c-kube-api-access-vz59m\") pod \"payload-pre-processing-e2e-worker-w5-544fc8-cb5b6cbd7-h7tjd\" (UID: \"97367526-d21f-45f0-bc3f-d36ae4a1fe7c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-544fc8-cb5b6cbd7-h7tjd"
Oct 30 19:30:52.895305 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:52.891833    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97367526-d21f-45f0-bc3f-d36ae4a1fe7c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-544fc8-cb5b6cbd7-h7tjd\" (UID: \"97367526-d21f-45f0-bc3f-d36ae4a1fe7c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-544fc8-cb5b6cbd7-h7tjd"
Oct 30 19:30:52.926674 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:52.926619    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vz59m\" (UniqueName: \"kubernetes.io/projected/97367526-d21f-45f0-bc3f-d36ae4a1fe7c-kube-api-access-vz59m\") pod \"payload-pre-processing-e2e-worker-w5-544fc8-cb5b6cbd7-h7tjd\" (UID: \"97367526-d21f-45f0-bc3f-d36ae4a1fe7c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-544fc8-cb5b6cbd7-h7tjd"
Oct 30 19:30:52.990922 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:52.990781    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ef9a5a-796b5cf9dc-kmcg7" event={"ID":"3b797e1f-bd98-4740-b15d-6ee4e1029d2a","Type":"ContainerStarted","Data":"d3a10aa6120cd43ac32dcff62999d39a94e839e75b0014873b4612e426663b6b"}
Oct 30 19:30:52.990922 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:52.990847    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ef9a5a-796b5cf9dc-kmcg7"
Oct 30 19:30:52.990922 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:52.990912    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ef9a5a-796b5cf9dc-kmcg7" event={"ID":"3b797e1f-bd98-4740-b15d-6ee4e1029d2a","Type":"ContainerStarted","Data":"5d5e530766e0c63fab7e6fe79da5bdd98b52907f3c32c3254dc0408d152ae067"}
Oct 30 19:30:52.992393 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:52.992370    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-ef9a5a-59fc6bf976-zfzks" event={"ID":"09920b13-483e-47d3-89f0-c22163568979","Type":"ContainerStarted","Data":"874488b95670c3078fd206e79eb89fb510f58632d62b6a81e4c02f9372f85146"}
Oct 30 19:30:52.992489 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:52.992410    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-ef9a5a-59fc6bf976-zfzks" event={"ID":"09920b13-483e-47d3-89f0-c22163568979","Type":"ContainerStarted","Data":"4f9e9fc15c8a115c6748a53ff2a5dab38eb3240ab074fd016b9f1eee9d3a863d"}
Oct 30 19:30:52.992489 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:52.992434    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w6-ef9a5a-59fc6bf976-zfzks"
Oct 30 19:30:52.996036 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:52.996018    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/615c6e69-747b-4294-ae5b-e0958527002c-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-544fc8-78bc5bc8bd-wjdrd\" (UID: \"615c6e69-747b-4294-ae5b-e0958527002c\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-544fc8-78bc5bc8bd-wjdrd"
Oct 30 19:30:52.996106 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:52.996045    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jprj8\" (UniqueName: \"kubernetes.io/projected/615c6e69-747b-4294-ae5b-e0958527002c-kube-api-access-jprj8\") pod \"payload-processing-e2e-worker-w5-544fc8-78bc5bc8bd-wjdrd\" (UID: \"615c6e69-747b-4294-ae5b-e0958527002c\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-544fc8-78bc5bc8bd-wjdrd"
Oct 30 19:30:53.024976 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:53.024951    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-544fc8-cb5b6cbd7-h7tjd"
Oct 30 19:30:53.036174 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:53.036119    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ef9a5a-796b5cf9dc-kmcg7" podStartSLOduration=2.036102592 podStartE2EDuration="2.036102592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:53.035273084 +0000 UTC m=+564.878205024" watchObservedRunningTime="2026-10-30 19:30:53.036102592 +0000 UTC m=+564.879034532"
Oct 30 19:30:53.078317 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:53.076690    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w6-ef9a5a-59fc6bf976-zfzks" podStartSLOduration=2.0766752569999998 podStartE2EDuration="2.076675257s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:53.076027319 +0000 UTC m=+564.918959285" watchObservedRunningTime="2026-10-30 19:30:53.076675257 +0000 UTC m=+564.919607197"
Oct 30 19:30:53.097875 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:53.097829    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/615c6e69-747b-4294-ae5b-e0958527002c-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-544fc8-78bc5bc8bd-wjdrd\" (UID: \"615c6e69-747b-4294-ae5b-e0958527002c\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-544fc8-78bc5bc8bd-wjdrd"
Oct 30 19:30:53.098127 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:53.098104    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jprj8\" (UniqueName: \"kubernetes.io/projected/615c6e69-747b-4294-ae5b-e0958527002c-kube-api-access-jprj8\") pod \"payload-processing-e2e-worker-w5-544fc8-78bc5bc8bd-wjdrd\" (UID: \"615c6e69-747b-4294-ae5b-e0958527002c\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-544fc8-78bc5bc8bd-wjdrd"
Oct 30 19:30:53.098601 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:53.098580    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/615c6e69-747b-4294-ae5b-e0958527002c-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-544fc8-78bc5bc8bd-wjdrd\" (UID: \"615c6e69-747b-4294-ae5b-e0958527002c\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-544fc8-78bc5bc8bd-wjdrd"
Oct 30 19:30:53.118949 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:53.118916    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jprj8\" (UniqueName: \"kubernetes.io/projected/615c6e69-747b-4294-ae5b-e0958527002c-kube-api-access-jprj8\") pod \"payload-processing-e2e-worker-w5-544fc8-78bc5bc8bd-wjdrd\" (UID: \"615c6e69-747b-4294-ae5b-e0958527002c\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-544fc8-78bc5bc8bd-wjdrd"
Oct 30 19:30:53.330146 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:53.329828    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-544fc8-cb5b6cbd7-h7tjd"]
Oct 30 19:30:53.334055 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:30:53.334020    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97367526_d21f_45f0_bc3f_d36ae4a1fe7c.slice/crio-bdf318f6875fb1c9ea525769ee1aeb658b844409c15aa17e3650c26b97f405d5 WatchSource:0}: Error finding container bdf318f6875fb1c9ea525769ee1aeb658b844409c15aa17e3650c26b97f405d5: Status 404 returned error can't find the container with id bdf318f6875fb1c9ea525769ee1aeb658b844409c15aa17e3650c26b97f405d5
Oct 30 19:30:53.416605 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:53.416570    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-544fc8-78bc5bc8bd-wjdrd"
Oct 30 19:30:53.542177 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:53.542144    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-dd23d1-6968f7db99-q6kfn"]
Oct 30 19:30:53.547120 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:53.547099    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd23d1-6968f7db99-q6kfn"
Oct 30 19:30:53.565301 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:53.565116    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-dd23d1\""
Oct 30 19:30:53.565301 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:53.565268    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-dd23d1-dockercfg-l5cj4\""
Oct 30 19:30:53.567427 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:53.567402    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-dd23d1-6968f7db99-q6kfn"]
Oct 30 19:30:53.625639 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:53.611940    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbjt7\" (UniqueName: \"kubernetes.io/projected/b2b39c26-0fce-4ba2-88cb-b171bb43c403-kube-api-access-rbjt7\") pod \"payload-pre-processing-e2e-worker-w3-dd23d1-6968f7db99-q6kfn\" (UID: \"b2b39c26-0fce-4ba2-88cb-b171bb43c403\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd23d1-6968f7db99-q6kfn"
Oct 30 19:30:53.625639 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:53.612016    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2b39c26-0fce-4ba2-88cb-b171bb43c403-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-dd23d1-6968f7db99-q6kfn\" (UID: \"b2b39c26-0fce-4ba2-88cb-b171bb43c403\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd23d1-6968f7db99-q6kfn"
Oct 30 19:30:53.645888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:53.638114    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-dd23d1-dddbddc84-htdcq"]
Oct 30 19:30:53.655317 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:53.655287    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-dd23d1-dddbddc84-htdcq"
Oct 30 19:30:53.673228 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:53.673194    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-dd23d1-dddbddc84-htdcq"]
Oct 30 19:30:53.720622 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:53.720574    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frjvc\" (UniqueName: \"kubernetes.io/projected/b876f230-c55f-4a4e-bf7d-c6d728ab8be3-kube-api-access-frjvc\") pod \"payload-processing-e2e-worker-w3-dd23d1-dddbddc84-htdcq\" (UID: \"b876f230-c55f-4a4e-bf7d-c6d728ab8be3\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-dd23d1-dddbddc84-htdcq"
Oct 30 19:30:53.722041 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:53.721951    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbjt7\" (UniqueName: \"kubernetes.io/projected/b2b39c26-0fce-4ba2-88cb-b171bb43c403-kube-api-access-rbjt7\") pod \"payload-pre-processing-e2e-worker-w3-dd23d1-6968f7db99-q6kfn\" (UID: \"b2b39c26-0fce-4ba2-88cb-b171bb43c403\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd23d1-6968f7db99-q6kfn"
Oct 30 19:30:53.722243 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:53.722225    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2b39c26-0fce-4ba2-88cb-b171bb43c403-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-dd23d1-6968f7db99-q6kfn\" (UID: \"b2b39c26-0fce-4ba2-88cb-b171bb43c403\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd23d1-6968f7db99-q6kfn"
Oct 30 19:30:53.722437 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:53.722421    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b876f230-c55f-4a4e-bf7d-c6d728ab8be3-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-dd23d1-dddbddc84-htdcq\" (UID: \"b876f230-c55f-4a4e-bf7d-c6d728ab8be3\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-dd23d1-dddbddc84-htdcq"
Oct 30 19:30:53.724605 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:53.723727    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2b39c26-0fce-4ba2-88cb-b171bb43c403-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-dd23d1-6968f7db99-q6kfn\" (UID: \"b2b39c26-0fce-4ba2-88cb-b171bb43c403\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd23d1-6968f7db99-q6kfn"
Oct 30 19:30:53.737807 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:53.737760    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-544fc8-78bc5bc8bd-wjdrd"]
Oct 30 19:30:53.743580 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:30:53.743551    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod615c6e69_747b_4294_ae5b_e0958527002c.slice/crio-8a9b4e0d1c69c496eff5a07ae98b46b7ddf0018707e8f7ab9892ef3c1698bcd2 WatchSource:0}: Error finding container 8a9b4e0d1c69c496eff5a07ae98b46b7ddf0018707e8f7ab9892ef3c1698bcd2: Status 404 returned error can't find the container with id 8a9b4e0d1c69c496eff5a07ae98b46b7ddf0018707e8f7ab9892ef3c1698bcd2
Oct 30 19:30:53.745536 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:53.745332    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbjt7\" (UniqueName: \"kubernetes.io/projected/b2b39c26-0fce-4ba2-88cb-b171bb43c403-kube-api-access-rbjt7\") pod \"payload-pre-processing-e2e-worker-w3-dd23d1-6968f7db99-q6kfn\" (UID: \"b2b39c26-0fce-4ba2-88cb-b171bb43c403\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd23d1-6968f7db99-q6kfn"
Oct 30 19:30:53.823954 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:53.823918    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frjvc\" (UniqueName: \"kubernetes.io/projected/b876f230-c55f-4a4e-bf7d-c6d728ab8be3-kube-api-access-frjvc\") pod \"payload-processing-e2e-worker-w3-dd23d1-dddbddc84-htdcq\" (UID: \"b876f230-c55f-4a4e-bf7d-c6d728ab8be3\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-dd23d1-dddbddc84-htdcq"
Oct 30 19:30:53.824091 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:53.824064    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b876f230-c55f-4a4e-bf7d-c6d728ab8be3-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-dd23d1-dddbddc84-htdcq\" (UID: \"b876f230-c55f-4a4e-bf7d-c6d728ab8be3\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-dd23d1-dddbddc84-htdcq"
Oct 30 19:30:53.824750 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:53.824724    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b876f230-c55f-4a4e-bf7d-c6d728ab8be3-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-dd23d1-dddbddc84-htdcq\" (UID: \"b876f230-c55f-4a4e-bf7d-c6d728ab8be3\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-dd23d1-dddbddc84-htdcq"
Oct 30 19:30:53.845266 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:53.845238    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frjvc\" (UniqueName: \"kubernetes.io/projected/b876f230-c55f-4a4e-bf7d-c6d728ab8be3-kube-api-access-frjvc\") pod \"payload-processing-e2e-worker-w3-dd23d1-dddbddc84-htdcq\" (UID: \"b876f230-c55f-4a4e-bf7d-c6d728ab8be3\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-dd23d1-dddbddc84-htdcq"
Oct 30 19:30:53.870295 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:53.870263    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd23d1-6968f7db99-q6kfn"
Oct 30 19:30:53.977504 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:53.977473    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-dd23d1-dddbddc84-htdcq"
Oct 30 19:30:54.001371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:54.001238    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-544fc8-78bc5bc8bd-wjdrd" event={"ID":"615c6e69-747b-4294-ae5b-e0958527002c","Type":"ContainerStarted","Data":"3403130a8c3692d69fa1926b8a30593c925449d9819d4695a00ac86607e73e44"}
Oct 30 19:30:54.001371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:54.001312    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w5-544fc8-78bc5bc8bd-wjdrd"
Oct 30 19:30:54.001371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:54.001340    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-544fc8-78bc5bc8bd-wjdrd" event={"ID":"615c6e69-747b-4294-ae5b-e0958527002c","Type":"ContainerStarted","Data":"8a9b4e0d1c69c496eff5a07ae98b46b7ddf0018707e8f7ab9892ef3c1698bcd2"}
Oct 30 19:30:54.003461 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:54.003411    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-544fc8-cb5b6cbd7-h7tjd" event={"ID":"97367526-d21f-45f0-bc3f-d36ae4a1fe7c","Type":"ContainerStarted","Data":"88c377fab912723f7bc47ba8f7f4be48d4fc6b75a265068abf827009a1c23094"}
Oct 30 19:30:54.003588 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:54.003473    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-544fc8-cb5b6cbd7-h7tjd" event={"ID":"97367526-d21f-45f0-bc3f-d36ae4a1fe7c","Type":"ContainerStarted","Data":"bdf318f6875fb1c9ea525769ee1aeb658b844409c15aa17e3650c26b97f405d5"}
Oct 30 19:30:54.003588 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:54.003533    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-544fc8-cb5b6cbd7-h7tjd"
Oct 30 19:30:54.044821 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:54.044593    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w5-544fc8-78bc5bc8bd-wjdrd" podStartSLOduration=2.044574171 podStartE2EDuration="2.044574171s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:54.042935595 +0000 UTC m=+565.885867535" watchObservedRunningTime="2026-10-30 19:30:54.044574171 +0000 UTC m=+565.887506108"
Oct 30 19:30:54.088619 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:54.088563    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-544fc8-cb5b6cbd7-h7tjd" podStartSLOduration=2.088545991 podStartE2EDuration="2.088545991s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:54.086913927 +0000 UTC m=+565.929845867" watchObservedRunningTime="2026-10-30 19:30:54.088545991 +0000 UTC m=+565.931477930"
Oct 30 19:30:54.173570 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:54.173540    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-dd23d1-6968f7db99-q6kfn"]
Oct 30 19:30:54.175738 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:30:54.175713    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2b39c26_0fce_4ba2_88cb_b171bb43c403.slice/crio-f273d2fd6c5aa39d932d025fb085b83bd52c61f0b5a4953d5f34d2917e36b498 WatchSource:0}: Error finding container f273d2fd6c5aa39d932d025fb085b83bd52c61f0b5a4953d5f34d2917e36b498: Status 404 returned error can't find the container with id f273d2fd6c5aa39d932d025fb085b83bd52c61f0b5a4953d5f34d2917e36b498
Oct 30 19:30:54.292903 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:54.292881    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-dd23d1-dddbddc84-htdcq"]
Oct 30 19:30:54.295619 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:30:54.295585    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb876f230_c55f_4a4e_bf7d_c6d728ab8be3.slice/crio-84d2b2180a873a1a1502b7dc707a776b572823bb27a947e103002e6d54a9b4ff WatchSource:0}: Error finding container 84d2b2180a873a1a1502b7dc707a776b572823bb27a947e103002e6d54a9b4ff: Status 404 returned error can't find the container with id 84d2b2180a873a1a1502b7dc707a776b572823bb27a947e103002e6d54a9b4ff
Oct 30 19:30:54.888002 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:54.887972    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:30:55.010375 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:55.010328    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd23d1-6968f7db99-q6kfn" event={"ID":"b2b39c26-0fce-4ba2-88cb-b171bb43c403","Type":"ContainerStarted","Data":"908c7df9924040b4f2880adea9508d607f52aff4d69393f6376d0251dce243cb"}
Oct 30 19:30:55.010827 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:55.010401    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd23d1-6968f7db99-q6kfn" event={"ID":"b2b39c26-0fce-4ba2-88cb-b171bb43c403","Type":"ContainerStarted","Data":"f273d2fd6c5aa39d932d025fb085b83bd52c61f0b5a4953d5f34d2917e36b498"}
Oct 30 19:30:55.010827 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:55.010434    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd23d1-6968f7db99-q6kfn"
Oct 30 19:30:55.012102 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:55.012070    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-dd23d1-dddbddc84-htdcq" event={"ID":"b876f230-c55f-4a4e-bf7d-c6d728ab8be3","Type":"ContainerStarted","Data":"5343c62f8759e651043827b84dff07c6ef3de56bc945d10b3bba8e1ad7486780"}
Oct 30 19:30:55.012237 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:55.012121    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-dd23d1-dddbddc84-htdcq" event={"ID":"b876f230-c55f-4a4e-bf7d-c6d728ab8be3","Type":"ContainerStarted","Data":"84d2b2180a873a1a1502b7dc707a776b572823bb27a947e103002e6d54a9b4ff"}
Oct 30 19:30:55.012404 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:55.012389    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-dd23d1-dddbddc84-htdcq"
Oct 30 19:30:55.065583 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:55.065518    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd23d1-6968f7db99-q6kfn" podStartSLOduration=2.065498744 podStartE2EDuration="2.065498744s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:55.063615553 +0000 UTC m=+566.906547492" watchObservedRunningTime="2026-10-30 19:30:55.065498744 +0000 UTC m=+566.908430685"
Oct 30 19:30:55.106033 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:55.105967    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w3-dd23d1-dddbddc84-htdcq" podStartSLOduration=2.105943009 podStartE2EDuration="2.105943009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:55.104606353 +0000 UTC m=+566.947538292" watchObservedRunningTime="2026-10-30 19:30:55.105943009 +0000 UTC m=+566.948874978"
Oct 30 19:30:55.840386 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:55.840354    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-3735b2-6f754b9b8b-b9rgh"]
Oct 30 19:30:55.844777 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:55.844756    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-3735b2-6f754b9b8b-b9rgh"
Oct 30 19:30:55.879312 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:55.879285    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w2-3735b2\""
Oct 30 19:30:55.881440 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:55.881415    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-3735b2-6f754b9b8b-b9rgh"]
Oct 30 19:30:55.897676 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:55.897645    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-3735b2-795c89ffc9-rtdm4"]
Oct 30 19:30:55.901798 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:55.901777    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-3735b2-795c89ffc9-rtdm4"
Oct 30 19:30:55.929696 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:55.929668    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-3735b2-795c89ffc9-rtdm4"]
Oct 30 19:30:55.958658 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:55.958619    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w2-3735b2-dockercfg-h4d6p\""
Oct 30 19:30:55.973141 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:55.973105    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1714315-7df0-4860-8759-bdfab7c13824-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-3735b2-6f754b9b8b-b9rgh\" (UID: \"b1714315-7df0-4860-8759-bdfab7c13824\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-3735b2-6f754b9b8b-b9rgh"
Oct 30 19:30:55.973336 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:55.973171    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-868pq\" (UniqueName: \"kubernetes.io/projected/b1714315-7df0-4860-8759-bdfab7c13824-kube-api-access-868pq\") pod \"payload-processing-e2e-worker-w2-3735b2-6f754b9b8b-b9rgh\" (UID: \"b1714315-7df0-4860-8759-bdfab7c13824\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-3735b2-6f754b9b8b-b9rgh"
Oct 30 19:30:55.973336 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:55.973284    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97e31b29-cb25-492c-bd66-fcab4b422ab9-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-3735b2-795c89ffc9-rtdm4\" (UID: \"97e31b29-cb25-492c-bd66-fcab4b422ab9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-3735b2-795c89ffc9-rtdm4"
Oct 30 19:30:55.973458 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:55.973425    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27bqp\" (UniqueName: \"kubernetes.io/projected/97e31b29-cb25-492c-bd66-fcab4b422ab9-kube-api-access-27bqp\") pod \"payload-pre-processing-e2e-worker-w2-3735b2-795c89ffc9-rtdm4\" (UID: \"97e31b29-cb25-492c-bd66-fcab4b422ab9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-3735b2-795c89ffc9-rtdm4"
Oct 30 19:30:56.075126 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:56.075070    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97e31b29-cb25-492c-bd66-fcab4b422ab9-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-3735b2-795c89ffc9-rtdm4\" (UID: \"97e31b29-cb25-492c-bd66-fcab4b422ab9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-3735b2-795c89ffc9-rtdm4"
Oct 30 19:30:56.075758 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:56.075232    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27bqp\" (UniqueName: \"kubernetes.io/projected/97e31b29-cb25-492c-bd66-fcab4b422ab9-kube-api-access-27bqp\") pod \"payload-pre-processing-e2e-worker-w2-3735b2-795c89ffc9-rtdm4\" (UID: \"97e31b29-cb25-492c-bd66-fcab4b422ab9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-3735b2-795c89ffc9-rtdm4"
Oct 30 19:30:56.075758 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:56.075309    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1714315-7df0-4860-8759-bdfab7c13824-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-3735b2-6f754b9b8b-b9rgh\" (UID: \"b1714315-7df0-4860-8759-bdfab7c13824\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-3735b2-6f754b9b8b-b9rgh"
Oct 30 19:30:56.075758 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:56.075416    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-868pq\" (UniqueName: \"kubernetes.io/projected/b1714315-7df0-4860-8759-bdfab7c13824-kube-api-access-868pq\") pod \"payload-processing-e2e-worker-w2-3735b2-6f754b9b8b-b9rgh\" (UID: \"b1714315-7df0-4860-8759-bdfab7c13824\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-3735b2-6f754b9b8b-b9rgh"
Oct 30 19:30:56.079283 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:56.078585    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97e31b29-cb25-492c-bd66-fcab4b422ab9-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-3735b2-795c89ffc9-rtdm4\" (UID: \"97e31b29-cb25-492c-bd66-fcab4b422ab9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-3735b2-795c89ffc9-rtdm4"
Oct 30 19:30:56.080916 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:56.080767    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1714315-7df0-4860-8759-bdfab7c13824-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-3735b2-6f754b9b8b-b9rgh\" (UID: \"b1714315-7df0-4860-8759-bdfab7c13824\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-3735b2-6f754b9b8b-b9rgh"
Oct 30 19:30:56.109939 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:56.106423    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-868pq\" (UniqueName: \"kubernetes.io/projected/b1714315-7df0-4860-8759-bdfab7c13824-kube-api-access-868pq\") pod \"payload-processing-e2e-worker-w2-3735b2-6f754b9b8b-b9rgh\" (UID: \"b1714315-7df0-4860-8759-bdfab7c13824\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-3735b2-6f754b9b8b-b9rgh"
Oct 30 19:30:56.146427 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:56.145898    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27bqp\" (UniqueName: \"kubernetes.io/projected/97e31b29-cb25-492c-bd66-fcab4b422ab9-kube-api-access-27bqp\") pod \"payload-pre-processing-e2e-worker-w2-3735b2-795c89ffc9-rtdm4\" (UID: \"97e31b29-cb25-492c-bd66-fcab4b422ab9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-3735b2-795c89ffc9-rtdm4"
Oct 30 19:30:56.157543 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:56.157506    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-3735b2-6f754b9b8b-b9rgh"
Oct 30 19:30:56.215574 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:56.215481    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-3735b2-795c89ffc9-rtdm4"
Oct 30 19:30:56.465677 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:56.465653    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-3735b2-6f754b9b8b-b9rgh"]
Oct 30 19:30:56.467936 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:30:56.467909    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1714315_7df0_4860_8759_bdfab7c13824.slice/crio-b2c7ecaf9ef6ca845b30ef6f719e980734c4f6a6d2f2eb37762de0c312b3f412 WatchSource:0}: Error finding container b2c7ecaf9ef6ca845b30ef6f719e980734c4f6a6d2f2eb37762de0c312b3f412: Status 404 returned error can't find the container with id b2c7ecaf9ef6ca845b30ef6f719e980734c4f6a6d2f2eb37762de0c312b3f412
Oct 30 19:30:56.530348 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:56.530323    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-3735b2-795c89ffc9-rtdm4"]
Oct 30 19:30:56.531756 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:30:56.531732    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97e31b29_cb25_492c_bd66_fcab4b422ab9.slice/crio-1a238b64f3e47159b3c1f69f2fbdc948da75be16aebccb18028ea527eef2efa5 WatchSource:0}: Error finding container 1a238b64f3e47159b3c1f69f2fbdc948da75be16aebccb18028ea527eef2efa5: Status 404 returned error can't find the container with id 1a238b64f3e47159b3c1f69f2fbdc948da75be16aebccb18028ea527eef2efa5
Oct 30 19:30:57.026216 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:57.026183    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-3735b2-6f754b9b8b-b9rgh" event={"ID":"b1714315-7df0-4860-8759-bdfab7c13824","Type":"ContainerStarted","Data":"4997148dc75040d79541acda6e57517025cdce6a421afddc7216ba87cb6f8b43"}
Oct 30 19:30:57.026394 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:57.026239    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-3735b2-6f754b9b8b-b9rgh" event={"ID":"b1714315-7df0-4860-8759-bdfab7c13824","Type":"ContainerStarted","Data":"b2c7ecaf9ef6ca845b30ef6f719e980734c4f6a6d2f2eb37762de0c312b3f412"}
Oct 30 19:30:57.026394 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:57.026334    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-3735b2-6f754b9b8b-b9rgh"
Oct 30 19:30:57.027773 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:57.027745    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-3735b2-795c89ffc9-rtdm4" event={"ID":"97e31b29-cb25-492c-bd66-fcab4b422ab9","Type":"ContainerStarted","Data":"69a5752e0cc2ce257d207fec4bb9b500ae53910687a54933a950146d925642e3"}
Oct 30 19:30:57.027907 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:57.027795    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-3735b2-795c89ffc9-rtdm4" event={"ID":"97e31b29-cb25-492c-bd66-fcab4b422ab9","Type":"ContainerStarted","Data":"1a238b64f3e47159b3c1f69f2fbdc948da75be16aebccb18028ea527eef2efa5"}
Oct 30 19:30:57.027907 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:57.027884    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-3735b2-795c89ffc9-rtdm4"
Oct 30 19:30:57.068975 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:57.068929    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w2-3735b2-6f754b9b8b-b9rgh" podStartSLOduration=2.068914538 podStartE2EDuration="2.068914538s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:57.06744391 +0000 UTC m=+568.910375849" watchObservedRunningTime="2026-10-30 19:30:57.068914538 +0000 UTC m=+568.911846477"
Oct 30 19:30:57.107762 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:57.107679    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-3735b2-795c89ffc9-rtdm4" podStartSLOduration=2.107663725 podStartE2EDuration="2.107663725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:57.107205264 +0000 UTC m=+568.950137203" watchObservedRunningTime="2026-10-30 19:30:57.107663725 +0000 UTC m=+568.950595663"
Oct 30 19:30:57.670540 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:57.670496    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-4b4c34-556f66b87f-d864z"]
Oct 30 19:30:57.674896 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:57.674842    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b4c34-556f66b87f-d864z"
Oct 30 19:30:57.693627 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:57.693606    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-4b4c34-dockercfg-6hk4x\""
Oct 30 19:30:57.703791 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:57.703764    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-4b4c34\""
Oct 30 19:30:57.705820 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:57.705798    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-4b4c34-556f66b87f-d864z"]
Oct 30 19:30:57.773401 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:57.773353    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-4b4c34-5bb88fdc94-s44w4"]
Oct 30 19:30:57.806918 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:57.804658    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znxqt\" (UniqueName: \"kubernetes.io/projected/b9fbde13-05e2-47a9-93bc-65cbd47efa6a-kube-api-access-znxqt\") pod \"payload-pre-processing-e2e-worker-w0-4b4c34-556f66b87f-d864z\" (UID: \"b9fbde13-05e2-47a9-93bc-65cbd47efa6a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b4c34-556f66b87f-d864z"
Oct 30 19:30:57.806918 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:57.804742    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9fbde13-05e2-47a9-93bc-65cbd47efa6a-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-4b4c34-556f66b87f-d864z\" (UID: \"b9fbde13-05e2-47a9-93bc-65cbd47efa6a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b4c34-556f66b87f-d864z"
Oct 30 19:30:57.810424 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:57.810396    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-4b4c34-5bb88fdc94-s44w4"]
Oct 30 19:30:57.810751 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:57.810686    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-4b4c34-5bb88fdc94-s44w4"
Oct 30 19:30:57.905983 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:57.905944    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znxqt\" (UniqueName: \"kubernetes.io/projected/b9fbde13-05e2-47a9-93bc-65cbd47efa6a-kube-api-access-znxqt\") pod \"payload-pre-processing-e2e-worker-w0-4b4c34-556f66b87f-d864z\" (UID: \"b9fbde13-05e2-47a9-93bc-65cbd47efa6a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b4c34-556f66b87f-d864z"
Oct 30 19:30:57.906213 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:57.906012    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9fbde13-05e2-47a9-93bc-65cbd47efa6a-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-4b4c34-556f66b87f-d864z\" (UID: \"b9fbde13-05e2-47a9-93bc-65cbd47efa6a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b4c34-556f66b87f-d864z"
Oct 30 19:30:57.906213 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:57.906054    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2dee0a84-130e-413d-8294-a86bf9c82c92-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-4b4c34-5bb88fdc94-s44w4\" (UID: \"2dee0a84-130e-413d-8294-a86bf9c82c92\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-4b4c34-5bb88fdc94-s44w4"
Oct 30 19:30:57.906213 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:57.906119    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbj8w\" (UniqueName: \"kubernetes.io/projected/2dee0a84-130e-413d-8294-a86bf9c82c92-kube-api-access-bbj8w\") pod \"payload-processing-e2e-worker-w0-4b4c34-5bb88fdc94-s44w4\" (UID: \"2dee0a84-130e-413d-8294-a86bf9c82c92\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-4b4c34-5bb88fdc94-s44w4"
Oct 30 19:30:57.906697 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:57.906675    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9fbde13-05e2-47a9-93bc-65cbd47efa6a-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-4b4c34-556f66b87f-d864z\" (UID: \"b9fbde13-05e2-47a9-93bc-65cbd47efa6a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b4c34-556f66b87f-d864z"
Oct 30 19:30:57.930317 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:57.930243    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-znxqt\" (UniqueName: \"kubernetes.io/projected/b9fbde13-05e2-47a9-93bc-65cbd47efa6a-kube-api-access-znxqt\") pod \"payload-pre-processing-e2e-worker-w0-4b4c34-556f66b87f-d864z\" (UID: \"b9fbde13-05e2-47a9-93bc-65cbd47efa6a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b4c34-556f66b87f-d864z"
Oct 30 19:30:57.988389 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:57.988355    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b4c34-556f66b87f-d864z"
Oct 30 19:30:58.007332 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:58.007288    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2dee0a84-130e-413d-8294-a86bf9c82c92-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-4b4c34-5bb88fdc94-s44w4\" (UID: \"2dee0a84-130e-413d-8294-a86bf9c82c92\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-4b4c34-5bb88fdc94-s44w4"
Oct 30 19:30:58.007521 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:58.007379    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbj8w\" (UniqueName: \"kubernetes.io/projected/2dee0a84-130e-413d-8294-a86bf9c82c92-kube-api-access-bbj8w\") pod \"payload-processing-e2e-worker-w0-4b4c34-5bb88fdc94-s44w4\" (UID: \"2dee0a84-130e-413d-8294-a86bf9c82c92\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-4b4c34-5bb88fdc94-s44w4"
Oct 30 19:30:58.008272 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:58.008144    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2dee0a84-130e-413d-8294-a86bf9c82c92-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-4b4c34-5bb88fdc94-s44w4\" (UID: \"2dee0a84-130e-413d-8294-a86bf9c82c92\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-4b4c34-5bb88fdc94-s44w4"
Oct 30 19:30:58.028731 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:58.028698    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbj8w\" (UniqueName: \"kubernetes.io/projected/2dee0a84-130e-413d-8294-a86bf9c82c92-kube-api-access-bbj8w\") pod \"payload-processing-e2e-worker-w0-4b4c34-5bb88fdc94-s44w4\" (UID: \"2dee0a84-130e-413d-8294-a86bf9c82c92\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-4b4c34-5bb88fdc94-s44w4"
Oct 30 19:30:58.129217 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:58.129178    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-4b4c34-5bb88fdc94-s44w4"
Oct 30 19:30:58.295297 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:58.295266    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-4b4c34-556f66b87f-d864z"]
Oct 30 19:30:58.297238 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:30:58.297209    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9fbde13_05e2_47a9_93bc_65cbd47efa6a.slice/crio-3be947d2a1f79c706b2b8cc3c16207e7e5eba66ab81ae5f4cb24a6b1524e6b76 WatchSource:0}: Error finding container 3be947d2a1f79c706b2b8cc3c16207e7e5eba66ab81ae5f4cb24a6b1524e6b76: Status 404 returned error can't find the container with id 3be947d2a1f79c706b2b8cc3c16207e7e5eba66ab81ae5f4cb24a6b1524e6b76
Oct 30 19:30:58.452266 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:58.452173    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-4b4c34-5bb88fdc94-s44w4"]
Oct 30 19:30:58.461054 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:30:58.461008    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dee0a84_130e_413d_8294_a86bf9c82c92.slice/crio-5879e75c5572c759f99d2f47fef1e161f92b647d943474c8a6a997b6f3bc80b0 WatchSource:0}: Error finding container 5879e75c5572c759f99d2f47fef1e161f92b647d943474c8a6a997b6f3bc80b0: Status 404 returned error can't find the container with id 5879e75c5572c759f99d2f47fef1e161f92b647d943474c8a6a997b6f3bc80b0
Oct 30 19:30:59.041390 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:59.041351    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b4c34-556f66b87f-d864z" event={"ID":"b9fbde13-05e2-47a9-93bc-65cbd47efa6a","Type":"ContainerStarted","Data":"e3f6c6e636856fe207c94d3ce951e8abfe0e683aecc943afd6d2dcbf073e792d"}
Oct 30 19:30:59.041594 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:59.041424    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b4c34-556f66b87f-d864z"
Oct 30 19:30:59.041594 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:59.041447    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b4c34-556f66b87f-d864z" event={"ID":"b9fbde13-05e2-47a9-93bc-65cbd47efa6a","Type":"ContainerStarted","Data":"3be947d2a1f79c706b2b8cc3c16207e7e5eba66ab81ae5f4cb24a6b1524e6b76"}
Oct 30 19:30:59.043123 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:59.043087    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-4b4c34-5bb88fdc94-s44w4" event={"ID":"2dee0a84-130e-413d-8294-a86bf9c82c92","Type":"ContainerStarted","Data":"a312f454db6b8f05a75ca06ed4c412547765da1bc3784654cb3ecf916aada2b5"}
Oct 30 19:30:59.043255 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:59.043158    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-4b4c34-5bb88fdc94-s44w4"
Oct 30 19:30:59.043255 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:59.043182    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-4b4c34-5bb88fdc94-s44w4" event={"ID":"2dee0a84-130e-413d-8294-a86bf9c82c92","Type":"ContainerStarted","Data":"5879e75c5572c759f99d2f47fef1e161f92b647d943474c8a6a997b6f3bc80b0"}
Oct 30 19:30:59.132064 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:59.131994    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-4b4c34-5bb88fdc94-s44w4" podStartSLOduration=2.131976023 podStartE2EDuration="2.131976023s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:59.130816375 +0000 UTC m=+570.973748314" watchObservedRunningTime="2026-10-30 19:30:59.131976023 +0000 UTC m=+570.974907963"
Oct 30 19:30:59.133286 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:30:59.133251    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b4c34-556f66b87f-d864z" podStartSLOduration=2.133240149 podStartE2EDuration="2.133240149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:59.086786756 +0000 UTC m=+570.929718697" watchObservedRunningTime="2026-10-30 19:30:59.133240149 +0000 UTC m=+570.976172165"
Oct 30 19:31:00.473675 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:00.473631    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:31:01.987446 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:01.987420    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-d797d2-67d78678ff-t4ccl"
Oct 30 19:31:04.006571 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:04.006543    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w6-ef9a5a-59fc6bf976-zfzks"
Oct 30 19:31:04.006969 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:04.006613    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ef9a5a-796b5cf9dc-kmcg7"
Oct 30 19:31:05.036367 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:05.036333    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w5-544fc8-78bc5bc8bd-wjdrd"
Oct 30 19:31:05.036896 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:05.036608    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-544fc8-cb5b6cbd7-h7tjd"
Oct 30 19:31:06.037534 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:06.037500    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd23d1-6968f7db99-q6kfn"
Oct 30 19:31:06.037984 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:06.037578    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-dd23d1-dddbddc84-htdcq"
Oct 30 19:31:06.409199 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:06.409164    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:31:08.074381 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:08.074353    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-3735b2-6f754b9b8b-b9rgh"
Oct 30 19:31:08.082823 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:08.081910    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-3735b2-795c89ffc9-rtdm4"
Oct 30 19:31:10.062429 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:10.062395    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b4c34-556f66b87f-d864z"
Oct 30 19:31:10.066486 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:10.066047    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-4b4c34-5bb88fdc94-s44w4"
Oct 30 19:31:13.251837 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:13.251747    5907 generic.go:356] "Generic (PLEG): container finished" podID="282f4fd8-512c-41b9-9989-0b84f2de1ce5" containerID="01633586736e0aaf824342a7876015a1a00880bfcafb68c4caa6f9f362652fd1" exitCode=0
Oct 30 19:31:13.251837 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:13.251815    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc" event={"ID":"282f4fd8-512c-41b9-9989-0b84f2de1ce5","Type":"ContainerDied","Data":"01633586736e0aaf824342a7876015a1a00880bfcafb68c4caa6f9f362652fd1"}
Oct 30 19:31:13.252304 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:13.252290    5907 scope.go:122] "RemoveContainer" containerID="01633586736e0aaf824342a7876015a1a00880bfcafb68c4caa6f9f362652fd1"
Oct 30 19:31:13.253784 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:13.253755    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:31:13.253854 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:13.253824    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:31:13.253922 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:13.253877    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:31:14.261047 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:14.260987    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc" event={"ID":"282f4fd8-512c-41b9-9989-0b84f2de1ce5","Type":"ContainerStarted","Data":"269422201e7a715b94c4696784ba9bc6fc8a305ce1ef1a2aef55601d94590ecf"}
Oct 30 19:31:14.735887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:14.735045    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:31:14.735887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:14.735212    5907 patch_prober.go:29] interesting pod/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 30 19:31:14.735887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:14.735265    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc" podUID="282f4fd8-512c-41b9-9989-0b84f2de1ce5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 30 19:31:15.617904 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:15.610886    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:31:15.736329 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:15.736277    5907 patch_prober.go:29] interesting pod/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 30 19:31:15.736527 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:15.736346    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc" podUID="282f4fd8-512c-41b9-9989-0b84f2de1ce5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 30 19:31:16.737373 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:16.737322    5907 patch_prober.go:29] interesting pod/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 30 19:31:16.737930 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:16.737390    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc" podUID="282f4fd8-512c-41b9-9989-0b84f2de1ce5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 30 19:31:16.950335 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:16.950295    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:31:16.988070 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:16.987983    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:31:16.988070 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:16.988041    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk" podUID="1b70d17b-10c5-4886-a033-caef763f1406" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:17.507338 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:31:17.507290    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:31:17.735729 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:17.735261    5907 patch_prober.go:29] interesting pod/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 30 19:31:17.735729 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:17.735330    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc" podUID="282f4fd8-512c-41b9-9989-0b84f2de1ce5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 30 19:31:17.950904 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:17.950771    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:31:17.950904 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:17.950827    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl" podUID="ca6dddc7-2d29-4500-ab3f-7c982fc8064d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:31:18.289055 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:18.289024    5907 generic.go:356] "Generic (PLEG): container finished" podID="ca6dddc7-2d29-4500-ab3f-7c982fc8064d" containerID="e0b5013aeda8e3120b40f658b4ca5bbce6b6b3cd404cf13c3c06bdd26c6d88e7" exitCode=0
Oct 30 19:31:18.289295 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:18.289094    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl" event={"ID":"ca6dddc7-2d29-4500-ab3f-7c982fc8064d","Type":"ContainerDied","Data":"e0b5013aeda8e3120b40f658b4ca5bbce6b6b3cd404cf13c3c06bdd26c6d88e7"}
Oct 30 19:31:18.289484 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:18.289464    5907 scope.go:122] "RemoveContainer" containerID="e0b5013aeda8e3120b40f658b4ca5bbce6b6b3cd404cf13c3c06bdd26c6d88e7"
Oct 30 19:31:18.290736 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:18.290704    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:31:18.290848 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:18.290739    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk" event={"ID":"1b70d17b-10c5-4886-a033-caef763f1406","Type":"ContainerDied","Data":"c34293f861a664a7de82259386c8ba601b042478b12ea37bf1d692111d7a5349"}
Oct 30 19:31:18.290848 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:18.290792    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:31:18.290848 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:18.290722    5907 generic.go:356] "Generic (PLEG): container finished" podID="1b70d17b-10c5-4886-a033-caef763f1406" containerID="c34293f861a664a7de82259386c8ba601b042478b12ea37bf1d692111d7a5349" exitCode=0
Oct 30 19:31:18.291041 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:18.290843    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:31:18.291041 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:18.291021    5907 scope.go:122] "RemoveContainer" containerID="c34293f861a664a7de82259386c8ba601b042478b12ea37bf1d692111d7a5349"
Oct 30 19:31:18.292238 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:18.292209    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:31:18.292339 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:18.292286    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:31:18.292339 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:18.292332    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:31:18.737654 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:18.735214    5907 patch_prober.go:29] interesting pod/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 30 19:31:18.737654 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:18.735271    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc" podUID="282f4fd8-512c-41b9-9989-0b84f2de1ce5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 30 19:31:19.331887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:19.331301    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl" event={"ID":"ca6dddc7-2d29-4500-ab3f-7c982fc8064d","Type":"ContainerStarted","Data":"1ba2acaa95a720fbb49c7b41d62bf366e9d93025619d73e65606f4c39afa6b91"}
Oct 30 19:31:19.356888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:19.352688    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk" event={"ID":"1b70d17b-10c5-4886-a033-caef763f1406","Type":"ContainerStarted","Data":"e9b111838bea8cfdc5ca9c46a1b78fe32a94c8e2c19c3a4cec0f2ead12f18ff4"}
Oct 30 19:31:19.463875 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:19.448446    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:31:19.485892 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:19.481057    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Oct 30 19:31:19.485892 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:19.481133    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk" podUID="1b70d17b-10c5-4886-a033-caef763f1406" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Oct 30 19:31:19.836886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:19.836123    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:31:19.853420 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:19.853331    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:31:20.075323 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:20.075285    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:31:20.104533 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:20.093290    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 30 19:31:20.105522 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:20.105228    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl" podUID="ca6dddc7-2d29-4500-ab3f-7c982fc8064d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 30 19:31:20.439885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:20.439265    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Oct 30 19:31:20.439885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:20.439326    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk" podUID="1b70d17b-10c5-4886-a033-caef763f1406" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Oct 30 19:31:21.071783 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:21.071748    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 30 19:31:21.071985 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:21.071808    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl" podUID="ca6dddc7-2d29-4500-ab3f-7c982fc8064d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 30 19:31:21.442882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:21.442271    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Oct 30 19:31:21.442882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:21.442337    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk" podUID="1b70d17b-10c5-4886-a033-caef763f1406" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Oct 30 19:31:22.075954 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:22.073324    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 30 19:31:22.075954 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:22.073393    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl" podUID="ca6dddc7-2d29-4500-ab3f-7c982fc8064d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 30 19:31:22.445310 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:22.445269    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Oct 30 19:31:22.446447 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:22.446414    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk" podUID="1b70d17b-10c5-4886-a033-caef763f1406" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Oct 30 19:31:23.071988 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:23.071956    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 30 19:31:23.072227 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:23.072002    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl" podUID="ca6dddc7-2d29-4500-ab3f-7c982fc8064d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 30 19:31:23.449900 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:23.447257    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Oct 30 19:31:23.449900 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:23.447334    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk" podUID="1b70d17b-10c5-4886-a033-caef763f1406" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Oct 30 19:31:24.031887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:24.029750    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:31:24.071439 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:24.071403    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 30 19:31:24.071642 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:24.071456    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl" podUID="ca6dddc7-2d29-4500-ab3f-7c982fc8064d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 30 19:31:24.442036 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:24.441999    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Oct 30 19:31:24.442343 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:24.442313    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk" podUID="1b70d17b-10c5-4886-a033-caef763f1406" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Oct 30 19:31:25.074882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:25.074835    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:31:25.075266 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:25.075141    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:31:25.075922 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:25.075906    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:31:25.442756 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:25.442724    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:31:25.443084 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:25.443062    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:31:25.443658 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:25.443643    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:31:35.296110 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:35.296069    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:31:46.511730 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:46.511696    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:31:47.538639 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:31:47.538598    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:31:58.301976 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:31:58.301932    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:32:08.802998 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:08.802790    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"]
Oct 30 19:32:08.808835 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:08.808814    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:32:08.827826 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:08.827801    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"]
Oct 30 19:32:08.838018 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:08.837986    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-3735b2\"/\"premium-w2-3735b2-kserve-self-signed-certs\""
Oct 30 19:32:08.838018 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:08.837988    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-3735b2\"/\"default-dockercfg-w7mcc\""
Oct 30 19:32:08.849938 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:08.849915    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-3735b2\"/\"openshift-service-ca.crt\""
Oct 30 19:32:08.850094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:08.849968    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-3735b2\"/\"kube-root-ca.crt\""
Oct 30 19:32:08.992027 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:08.991930    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-dshm\") pod \"premium-w2-3735b2-kserve-89b655654-2vsn7\" (UID: \"0832aa0a-f1ee-488f-a44e-4b56726e9b7f\") " pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:32:08.992414 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:08.992382    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt47b\" (UniqueName: \"kubernetes.io/projected/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-kube-api-access-pt47b\") pod \"premium-w2-3735b2-kserve-89b655654-2vsn7\" (UID: \"0832aa0a-f1ee-488f-a44e-4b56726e9b7f\") " pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:32:08.992543 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:08.992529    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-tls-certs\") pod \"premium-w2-3735b2-kserve-89b655654-2vsn7\" (UID: \"0832aa0a-f1ee-488f-a44e-4b56726e9b7f\") " pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:32:08.992648 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:08.992633    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-home\") pod \"premium-w2-3735b2-kserve-89b655654-2vsn7\" (UID: \"0832aa0a-f1ee-488f-a44e-4b56726e9b7f\") " pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:32:08.992807 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:08.992793    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-model-cache\") pod \"premium-w2-3735b2-kserve-89b655654-2vsn7\" (UID: \"0832aa0a-f1ee-488f-a44e-4b56726e9b7f\") " pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:32:08.992980 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:08.992965    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-tmp-dir\") pod \"premium-w2-3735b2-kserve-89b655654-2vsn7\" (UID: \"0832aa0a-f1ee-488f-a44e-4b56726e9b7f\") " pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:32:09.094797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:09.094718    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-tmp-dir\") pod \"premium-w2-3735b2-kserve-89b655654-2vsn7\" (UID: \"0832aa0a-f1ee-488f-a44e-4b56726e9b7f\") " pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:32:09.094797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:09.094794    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-dshm\") pod \"premium-w2-3735b2-kserve-89b655654-2vsn7\" (UID: \"0832aa0a-f1ee-488f-a44e-4b56726e9b7f\") " pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:32:09.095042 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:09.094840    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt47b\" (UniqueName: \"kubernetes.io/projected/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-kube-api-access-pt47b\") pod \"premium-w2-3735b2-kserve-89b655654-2vsn7\" (UID: \"0832aa0a-f1ee-488f-a44e-4b56726e9b7f\") " pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:32:09.095042 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:09.094888    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-tls-certs\") pod \"premium-w2-3735b2-kserve-89b655654-2vsn7\" (UID: \"0832aa0a-f1ee-488f-a44e-4b56726e9b7f\") " pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:32:09.095042 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:09.094908    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-home\") pod \"premium-w2-3735b2-kserve-89b655654-2vsn7\" (UID: \"0832aa0a-f1ee-488f-a44e-4b56726e9b7f\") " pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:32:09.095042 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:09.094968    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-model-cache\") pod \"premium-w2-3735b2-kserve-89b655654-2vsn7\" (UID: \"0832aa0a-f1ee-488f-a44e-4b56726e9b7f\") " pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:32:09.095272 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:09.095165    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-tmp-dir\") pod \"premium-w2-3735b2-kserve-89b655654-2vsn7\" (UID: \"0832aa0a-f1ee-488f-a44e-4b56726e9b7f\") " pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:32:09.095330 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:09.095304    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-home\") pod \"premium-w2-3735b2-kserve-89b655654-2vsn7\" (UID: \"0832aa0a-f1ee-488f-a44e-4b56726e9b7f\") " pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:32:09.095374 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:09.095327    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-model-cache\") pod \"premium-w2-3735b2-kserve-89b655654-2vsn7\" (UID: \"0832aa0a-f1ee-488f-a44e-4b56726e9b7f\") " pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:32:09.097202 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:09.097180    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-dshm\") pod \"premium-w2-3735b2-kserve-89b655654-2vsn7\" (UID: \"0832aa0a-f1ee-488f-a44e-4b56726e9b7f\") " pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:32:09.097439 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:09.097421    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-tls-certs\") pod \"premium-w2-3735b2-kserve-89b655654-2vsn7\" (UID: \"0832aa0a-f1ee-488f-a44e-4b56726e9b7f\") " pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:32:09.115787 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:09.115764    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt47b\" (UniqueName: \"kubernetes.io/projected/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-kube-api-access-pt47b\") pod \"premium-w2-3735b2-kserve-89b655654-2vsn7\" (UID: \"0832aa0a-f1ee-488f-a44e-4b56726e9b7f\") " pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:32:09.123639 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:09.123620    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:32:09.432018 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:09.431993    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"]
Oct 30 19:32:09.434285 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:32:09.434258    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0832aa0a_f1ee_488f_a44e_4b56726e9b7f.slice/crio-d7a442cbd6e4e78646252416795e2ed819806984c386d7709d8da71489efabe5 WatchSource:0}: Error finding container d7a442cbd6e4e78646252416795e2ed819806984c386d7709d8da71489efabe5: Status 404 returned error can't find the container with id d7a442cbd6e4e78646252416795e2ed819806984c386d7709d8da71489efabe5
Oct 30 19:32:09.651633 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:09.651592    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" event={"ID":"0832aa0a-f1ee-488f-a44e-4b56726e9b7f","Type":"ContainerStarted","Data":"740de951966db6ed02d8d9cf86d300a1468e8033e14afab8f08c503ab737e23c"}
Oct 30 19:32:09.651633 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:09.651641    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" event={"ID":"0832aa0a-f1ee-488f-a44e-4b56726e9b7f","Type":"ContainerStarted","Data":"d7a442cbd6e4e78646252416795e2ed819806984c386d7709d8da71489efabe5"}
Oct 30 19:32:09.694053 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:09.693945    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podStartSLOduration=1.693930868 podStartE2EDuration="1.693930868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:32:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:32:09.692067796 +0000 UTC m=+641.534999735" watchObservedRunningTime="2026-10-30 19:32:09.693930868 +0000 UTC m=+641.536862807"
Oct 30 19:32:10.124532 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:10.124505    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:32:11.963308 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:11.963266    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"]
Oct 30 19:32:11.968127 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:11.968102    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:32:11.986012 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:11.985984    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"]
Oct 30 19:32:11.996655 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:11.996613    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-4b4c34\"/\"openshift-service-ca.crt\""
Oct 30 19:32:11.996811 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:11.996680    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-4b4c34\"/\"premium-w0-4b4c34-kserve-self-signed-certs\""
Oct 30 19:32:11.996811 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:11.996695    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-4b4c34\"/\"kube-root-ca.crt\""
Oct 30 19:32:11.999528 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:11.999506    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-4b4c34\"/\"default-dockercfg-499bb\""
Oct 30 19:32:12.049310 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:12.049278    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f127fc10-e30f-405d-b5cf-be0d45cdee6e-model-cache\") pod \"premium-w0-4b4c34-kserve-5d84599686-zgzf8\" (UID: \"f127fc10-e30f-405d-b5cf-be0d45cdee6e\") " pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:32:12.049495 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:12.049327    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f127fc10-e30f-405d-b5cf-be0d45cdee6e-dshm\") pod \"premium-w0-4b4c34-kserve-5d84599686-zgzf8\" (UID: \"f127fc10-e30f-405d-b5cf-be0d45cdee6e\") " pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:32:12.049495 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:12.049425    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f127fc10-e30f-405d-b5cf-be0d45cdee6e-tmp-dir\") pod \"premium-w0-4b4c34-kserve-5d84599686-zgzf8\" (UID: \"f127fc10-e30f-405d-b5cf-be0d45cdee6e\") " pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:32:12.049495 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:12.049477    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvlj2\" (UniqueName: \"kubernetes.io/projected/f127fc10-e30f-405d-b5cf-be0d45cdee6e-kube-api-access-jvlj2\") pod \"premium-w0-4b4c34-kserve-5d84599686-zgzf8\" (UID: \"f127fc10-e30f-405d-b5cf-be0d45cdee6e\") " pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:32:12.049606 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:12.049522    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f127fc10-e30f-405d-b5cf-be0d45cdee6e-tls-certs\") pod \"premium-w0-4b4c34-kserve-5d84599686-zgzf8\" (UID: \"f127fc10-e30f-405d-b5cf-be0d45cdee6e\") " pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:32:12.049665 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:12.049641    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f127fc10-e30f-405d-b5cf-be0d45cdee6e-home\") pod \"premium-w0-4b4c34-kserve-5d84599686-zgzf8\" (UID: \"f127fc10-e30f-405d-b5cf-be0d45cdee6e\") " pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:32:12.164894 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:12.163448    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f127fc10-e30f-405d-b5cf-be0d45cdee6e-home\") pod \"premium-w0-4b4c34-kserve-5d84599686-zgzf8\" (UID: \"f127fc10-e30f-405d-b5cf-be0d45cdee6e\") " pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:32:12.164894 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:12.163598    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f127fc10-e30f-405d-b5cf-be0d45cdee6e-model-cache\") pod \"premium-w0-4b4c34-kserve-5d84599686-zgzf8\" (UID: \"f127fc10-e30f-405d-b5cf-be0d45cdee6e\") " pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:32:12.164894 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:12.163645    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f127fc10-e30f-405d-b5cf-be0d45cdee6e-dshm\") pod \"premium-w0-4b4c34-kserve-5d84599686-zgzf8\" (UID: \"f127fc10-e30f-405d-b5cf-be0d45cdee6e\") " pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:32:12.164894 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:12.163703    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f127fc10-e30f-405d-b5cf-be0d45cdee6e-tmp-dir\") pod \"premium-w0-4b4c34-kserve-5d84599686-zgzf8\" (UID: \"f127fc10-e30f-405d-b5cf-be0d45cdee6e\") " pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:32:12.164894 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:12.163760    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvlj2\" (UniqueName: \"kubernetes.io/projected/f127fc10-e30f-405d-b5cf-be0d45cdee6e-kube-api-access-jvlj2\") pod \"premium-w0-4b4c34-kserve-5d84599686-zgzf8\" (UID: \"f127fc10-e30f-405d-b5cf-be0d45cdee6e\") " pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:32:12.164894 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:12.163820    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f127fc10-e30f-405d-b5cf-be0d45cdee6e-tls-certs\") pod \"premium-w0-4b4c34-kserve-5d84599686-zgzf8\" (UID: \"f127fc10-e30f-405d-b5cf-be0d45cdee6e\") " pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:32:12.182884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:12.175590    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f127fc10-e30f-405d-b5cf-be0d45cdee6e-tmp-dir\") pod \"premium-w0-4b4c34-kserve-5d84599686-zgzf8\" (UID: \"f127fc10-e30f-405d-b5cf-be0d45cdee6e\") " pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:32:12.182884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:12.176271    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f127fc10-e30f-405d-b5cf-be0d45cdee6e-home\") pod \"premium-w0-4b4c34-kserve-5d84599686-zgzf8\" (UID: \"f127fc10-e30f-405d-b5cf-be0d45cdee6e\") " pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:32:12.182884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:12.176516    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f127fc10-e30f-405d-b5cf-be0d45cdee6e-model-cache\") pod \"premium-w0-4b4c34-kserve-5d84599686-zgzf8\" (UID: \"f127fc10-e30f-405d-b5cf-be0d45cdee6e\") " pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:32:12.193884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:12.191653    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f127fc10-e30f-405d-b5cf-be0d45cdee6e-tls-certs\") pod \"premium-w0-4b4c34-kserve-5d84599686-zgzf8\" (UID: \"f127fc10-e30f-405d-b5cf-be0d45cdee6e\") " pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:32:12.203148 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:12.203111    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f127fc10-e30f-405d-b5cf-be0d45cdee6e-dshm\") pod \"premium-w0-4b4c34-kserve-5d84599686-zgzf8\" (UID: \"f127fc10-e30f-405d-b5cf-be0d45cdee6e\") " pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:32:12.203342 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:12.203134    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvlj2\" (UniqueName: \"kubernetes.io/projected/f127fc10-e30f-405d-b5cf-be0d45cdee6e-kube-api-access-jvlj2\") pod \"premium-w0-4b4c34-kserve-5d84599686-zgzf8\" (UID: \"f127fc10-e30f-405d-b5cf-be0d45cdee6e\") " pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:32:12.284617 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:12.284535    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:32:12.591057 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:12.590979    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"]
Oct 30 19:32:12.594078 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:32:12.594053    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf127fc10_e30f_405d_b5cf_be0d45cdee6e.slice/crio-a26d7e7fcfb838edb8f326693dfd0e73cddf5578933a37759603b195745a8e60 WatchSource:0}: Error finding container a26d7e7fcfb838edb8f326693dfd0e73cddf5578933a37759603b195745a8e60: Status 404 returned error can't find the container with id a26d7e7fcfb838edb8f326693dfd0e73cddf5578933a37759603b195745a8e60
Oct 30 19:32:12.672756 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:12.672716    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" event={"ID":"f127fc10-e30f-405d-b5cf-be0d45cdee6e","Type":"ContainerStarted","Data":"02ae6121b780eea9dbffa8ba7a0ce00e619740331de6c77558feb4f596bf90f5"}
Oct 30 19:32:12.672928 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:12.672776    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" event={"ID":"f127fc10-e30f-405d-b5cf-be0d45cdee6e","Type":"ContainerStarted","Data":"a26d7e7fcfb838edb8f326693dfd0e73cddf5578933a37759603b195745a8e60"}
Oct 30 19:32:12.716202 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:12.716140    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podStartSLOduration=1.716094872 podStartE2EDuration="1.716094872s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:32:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:32:12.715448814 +0000 UTC m=+644.558380753" watchObservedRunningTime="2026-10-30 19:32:12.716094872 +0000 UTC m=+644.559026810"
Oct 30 19:32:13.285483 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:13.285442    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:32:14.316143 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:14.316112    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:32:17.500359 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:32:17.500323    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:32:19.123984 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:19.123943    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:32:19.137181 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:19.137148    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:32:19.725452 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:19.725423    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:32:22.296625 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:22.296589    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:32:22.304414 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:22.304390    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:32:22.742542 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:22.742511    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:32:39.611440 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:39.611406    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:32:44.387127 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:44.387083    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"]
Oct 30 19:32:44.391178 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:44.391152    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"
Oct 30 19:32:44.407084 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:44.407059    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-ef9a5a-openshift-default-dockercfg-dtqfz\""
Oct 30 19:32:44.423276 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:44.423243    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"]
Oct 30 19:32:44.480426 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:44.480392    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-credential-socket\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"
Oct 30 19:32:44.480653 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:44.480454    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p48gn\" (UniqueName: \"kubernetes.io/projected/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-kube-api-access-p48gn\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"
Oct 30 19:32:44.480653 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:44.480536    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-workload-socket\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"
Oct 30 19:32:44.480653 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:44.480575    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istiod-ca-cert\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"
Oct 30 19:32:44.480653 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:44.480641    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istio-token\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"
Oct 30 19:32:44.480849 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:44.480727    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istio-envoy\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"
Oct 30 19:32:44.480849 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:44.480742    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istio-data\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"
Oct 30 19:32:44.480849 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:44.480769    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-workload-certs\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"
Oct 30 19:32:44.480849 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:44.480835    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istio-podinfo\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"
Oct 30 19:32:44.582111 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:44.582075    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istio-envoy\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"
Oct 30 19:32:44.582111 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:44.582109    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istio-data\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"
Oct 30 19:32:44.582379 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:44.582250    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-workload-certs\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"
Oct 30 19:32:44.582379 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:44.582331    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istio-podinfo\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"
Oct 30 19:32:44.582487 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:44.582380    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-credential-socket\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"
Oct 30 19:32:44.582487 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:44.582458    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p48gn\" (UniqueName: \"kubernetes.io/projected/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-kube-api-access-p48gn\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"
Oct 30 19:32:44.582595 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:44.582488    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-workload-socket\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"
Oct 30 19:32:44.582595 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:44.582511    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istiod-ca-cert\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"
Oct 30 19:32:44.582595 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:44.582510    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istio-data\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"
Oct 30 19:32:44.582595 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:44.582579    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istio-token\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"
Oct 30 19:32:44.582806 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:44.582785    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-workload-certs\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"
Oct 30 19:32:44.583089 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:44.583065    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-workload-socket\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"
Oct 30 19:32:44.583335 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:44.583273    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-credential-socket\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"
Oct 30 19:32:44.583335 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:44.583290    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istiod-ca-cert\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"
Oct 30 19:32:44.585230 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:44.585203    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istio-podinfo\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"
Oct 30 19:32:44.585496 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:44.585465    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istio-envoy\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"
Oct 30 19:32:44.603898 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:44.603843    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istio-token\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"
Oct 30 19:32:44.604146 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:44.604130    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p48gn\" (UniqueName: \"kubernetes.io/projected/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-kube-api-access-p48gn\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"
Oct 30 19:32:44.704962 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:44.704877    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"
Oct 30 19:32:45.074257 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:45.074225    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"]
Oct 30 19:32:45.076023 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:32:45.075998    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01992c2f_0e71_4b86_8e8a_69bb7727bd6a.slice/crio-8b7e7664b1f7ccb6bd604f156a66dfbdd2e0913a422c0452a432cc015121a2e9 WatchSource:0}: Error finding container 8b7e7664b1f7ccb6bd604f156a66dfbdd2e0913a422c0452a432cc015121a2e9: Status 404 returned error can't find the container with id 8b7e7664b1f7ccb6bd604f156a66dfbdd2e0913a422c0452a432cc015121a2e9
Oct 30 19:32:45.078531 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:45.078491    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:32:45.078619 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:45.078567    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:32:45.078619 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:45.078596    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:32:45.862490 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:45.862452    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv" event={"ID":"01992c2f-0e71-4b86-8e8a-69bb7727bd6a","Type":"ContainerStarted","Data":"517f1f95ec7893dc438e29d7c93505719922128a86c72f062d50868d00705195"}
Oct 30 19:32:45.862954 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:45.862515    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv" event={"ID":"01992c2f-0e71-4b86-8e8a-69bb7727bd6a","Type":"ContainerStarted","Data":"8b7e7664b1f7ccb6bd604f156a66dfbdd2e0913a422c0452a432cc015121a2e9"}
Oct 30 19:32:46.705351 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:46.705313    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"
Oct 30 19:32:46.706935 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:46.706910    5907 patch_prober.go:29] interesting pod/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 19:32:46.707074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:46.706967    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv" podUID="01992c2f-0e71-4b86-8e8a-69bb7727bd6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 19:32:47.503820 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:32:47.503761    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:32:47.705801 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:47.705768    5907 patch_prober.go:29] interesting pod/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 19:32:47.705958 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:47.705816    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv" podUID="01992c2f-0e71-4b86-8e8a-69bb7727bd6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 19:32:48.705840 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:48.705809    5907 patch_prober.go:29] interesting pod/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 19:32:48.706333 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:48.705880    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv" podUID="01992c2f-0e71-4b86-8e8a-69bb7727bd6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 19:32:49.706086 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:49.706050    5907 patch_prober.go:29] interesting pod/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 19:32:49.706511 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:49.706105    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv" podUID="01992c2f-0e71-4b86-8e8a-69bb7727bd6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 19:32:50.425887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:50.410567    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv" podStartSLOduration=6.410547992 podStartE2EDuration="6.410547992s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:32:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:32:45.938303142 +0000 UTC m=+677.781235082" watchObservedRunningTime="2026-10-30 19:32:50.410547992 +0000 UTC m=+682.253479932"
Oct 30 19:32:50.432893 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:50.432125    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6b64d8bb-rpw59"]
Oct 30 19:32:50.437889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:50.437797    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b64d8bb-rpw59"
Oct 30 19:32:50.447200 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:50.447171    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6b64d8bb-rpw59"]
Oct 30 19:32:50.457541 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:50.457512    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:32:50.457653 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:50.457581    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:32:50.482574 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:50.482547    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8c883612-2ee9-49ed-a69d-d3dce0011061-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6b64d8bb-rpw59\" (UID: \"8c883612-2ee9-49ed-a69d-d3dce0011061\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b64d8bb-rpw59"
Oct 30 19:32:50.482705 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:50.482615    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8c883612-2ee9-49ed-a69d-d3dce0011061-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6b64d8bb-rpw59\" (UID: \"8c883612-2ee9-49ed-a69d-d3dce0011061\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b64d8bb-rpw59"
Oct 30 19:32:50.482765 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:50.482745    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjl6n\" (UniqueName: \"kubernetes.io/projected/8c883612-2ee9-49ed-a69d-d3dce0011061-kube-api-access-cjl6n\") pod \"maas-api-revoke-sub-6b64d8bb-rpw59\" (UID: \"8c883612-2ee9-49ed-a69d-d3dce0011061\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b64d8bb-rpw59"
Oct 30 19:32:50.584348 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:50.584312    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8c883612-2ee9-49ed-a69d-d3dce0011061-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6b64d8bb-rpw59\" (UID: \"8c883612-2ee9-49ed-a69d-d3dce0011061\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b64d8bb-rpw59"
Oct 30 19:32:50.584533 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:50.584401    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjl6n\" (UniqueName: \"kubernetes.io/projected/8c883612-2ee9-49ed-a69d-d3dce0011061-kube-api-access-cjl6n\") pod \"maas-api-revoke-sub-6b64d8bb-rpw59\" (UID: \"8c883612-2ee9-49ed-a69d-d3dce0011061\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b64d8bb-rpw59"
Oct 30 19:32:50.584533 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:50.584463    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8c883612-2ee9-49ed-a69d-d3dce0011061-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6b64d8bb-rpw59\" (UID: \"8c883612-2ee9-49ed-a69d-d3dce0011061\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b64d8bb-rpw59"
Oct 30 19:32:50.585137 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:50.585115    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8c883612-2ee9-49ed-a69d-d3dce0011061-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6b64d8bb-rpw59\" (UID: \"8c883612-2ee9-49ed-a69d-d3dce0011061\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b64d8bb-rpw59"
Oct 30 19:32:50.586920 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:50.586902    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8c883612-2ee9-49ed-a69d-d3dce0011061-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6b64d8bb-rpw59\" (UID: \"8c883612-2ee9-49ed-a69d-d3dce0011061\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b64d8bb-rpw59"
Oct 30 19:32:50.605837 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:50.605805    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjl6n\" (UniqueName: \"kubernetes.io/projected/8c883612-2ee9-49ed-a69d-d3dce0011061-kube-api-access-cjl6n\") pod \"maas-api-revoke-sub-6b64d8bb-rpw59\" (UID: \"8c883612-2ee9-49ed-a69d-d3dce0011061\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b64d8bb-rpw59"
Oct 30 19:32:50.705632 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:50.705533    5907 patch_prober.go:29] interesting pod/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 19:32:50.705632 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:50.705602    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv" podUID="01992c2f-0e71-4b86-8e8a-69bb7727bd6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 19:32:50.756732 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:50.756697    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b64d8bb-rpw59"
Oct 30 19:32:51.083038 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:51.083016    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6b64d8bb-rpw59"]
Oct 30 19:32:51.085583 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:32:51.085549    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c883612_2ee9_49ed_a69d_d3dce0011061.slice/crio-332c7a95faee925490896ce6866028cb9612581fad1cc9f88acde7ff02330528 WatchSource:0}: Error finding container 332c7a95faee925490896ce6866028cb9612581fad1cc9f88acde7ff02330528: Status 404 returned error can't find the container with id 332c7a95faee925490896ce6866028cb9612581fad1cc9f88acde7ff02330528
Oct 30 19:32:51.705333 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:51.705245    5907 patch_prober.go:29] interesting pod/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 19:32:51.705333 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:51.705293    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv" podUID="01992c2f-0e71-4b86-8e8a-69bb7727bd6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 19:32:51.902926 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:51.902887    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b64d8bb-rpw59" event={"ID":"8c883612-2ee9-49ed-a69d-d3dce0011061","Type":"ContainerStarted","Data":"bcfe545ac371653562bf23ba6af1c0118d245c7ffcd5bc3a34ed39fc1a0b6b58"}
Oct 30 19:32:51.903478 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:51.903006    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b64d8bb-rpw59" event={"ID":"8c883612-2ee9-49ed-a69d-d3dce0011061","Type":"ContainerStarted","Data":"332c7a95faee925490896ce6866028cb9612581fad1cc9f88acde7ff02330528"}
Oct 30 19:32:51.945003 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:51.944936    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b64d8bb-rpw59" podStartSLOduration=1.944890954 podStartE2EDuration="1.944890954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:32:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:32:51.943842973 +0000 UTC m=+683.786774912" watchObservedRunningTime="2026-10-30 19:32:51.944890954 +0000 UTC m=+683.787822894"
Oct 30 19:32:52.710333 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:52.710305    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"
Oct 30 19:32:52.710770 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:52.710732    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"
Oct 30 19:32:52.711536 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:52.711516    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"
Oct 30 19:32:52.911135 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:52.911100    5907 generic.go:356] "Generic (PLEG): container finished" podID="8c883612-2ee9-49ed-a69d-d3dce0011061" containerID="bcfe545ac371653562bf23ba6af1c0118d245c7ffcd5bc3a34ed39fc1a0b6b58" exitCode=0
Oct 30 19:32:52.911542 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:52.911236    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b64d8bb-rpw59" event={"ID":"8c883612-2ee9-49ed-a69d-d3dce0011061","Type":"ContainerDied","Data":"bcfe545ac371653562bf23ba6af1c0118d245c7ffcd5bc3a34ed39fc1a0b6b58"}
Oct 30 19:32:54.100903 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:54.078786    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b64d8bb-rpw59"
Oct 30 19:32:54.227318 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:54.227218    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8c883612-2ee9-49ed-a69d-d3dce0011061-kube-api-access-cjl6n\" (UniqueName: \"kubernetes.io/projected/8c883612-2ee9-49ed-a69d-d3dce0011061-kube-api-access-cjl6n\") pod \"8c883612-2ee9-49ed-a69d-d3dce0011061\" (UID: \"8c883612-2ee9-49ed-a69d-d3dce0011061\") "
Oct 30 19:32:54.227503 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:54.227382    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8c883612-2ee9-49ed-a69d-d3dce0011061-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8c883612-2ee9-49ed-a69d-d3dce0011061-maas-api-xks-ca\") pod \"8c883612-2ee9-49ed-a69d-d3dce0011061\" (UID: \"8c883612-2ee9-49ed-a69d-d3dce0011061\") "
Oct 30 19:32:54.227503 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:54.227427    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8c883612-2ee9-49ed-a69d-d3dce0011061-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8c883612-2ee9-49ed-a69d-d3dce0011061-maas-api-openshift-ca\") pod \"8c883612-2ee9-49ed-a69d-d3dce0011061\" (UID: \"8c883612-2ee9-49ed-a69d-d3dce0011061\") "
Oct 30 19:32:54.227821 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:54.227795    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c883612-2ee9-49ed-a69d-d3dce0011061-maas-api-openshift-ca" pod "8c883612-2ee9-49ed-a69d-d3dce0011061" (UID: "8c883612-2ee9-49ed-a69d-d3dce0011061"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:54.229467 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:54.229439    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c883612-2ee9-49ed-a69d-d3dce0011061-kube-api-access-cjl6n" pod "8c883612-2ee9-49ed-a69d-d3dce0011061" (UID: "8c883612-2ee9-49ed-a69d-d3dce0011061"). InnerVolumeSpecName "kube-api-access-cjl6n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:54.229592 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:54.229495    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c883612-2ee9-49ed-a69d-d3dce0011061-maas-api-xks-ca" pod "8c883612-2ee9-49ed-a69d-d3dce0011061" (UID: "8c883612-2ee9-49ed-a69d-d3dce0011061"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:54.329225 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:54.329191    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjl6n\" (UniqueName: \"kubernetes.io/projected/8c883612-2ee9-49ed-a69d-d3dce0011061-kube-api-access-cjl6n\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:32:54.329225 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:54.329222    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8c883612-2ee9-49ed-a69d-d3dce0011061-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:32:54.329225 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:54.329231    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8c883612-2ee9-49ed-a69d-d3dce0011061-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:32:54.924974 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:54.924945    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b64d8bb-rpw59"
Oct 30 19:32:54.925158 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:54.924952    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b64d8bb-rpw59" event={"ID":"8c883612-2ee9-49ed-a69d-d3dce0011061","Type":"ContainerDied","Data":"332c7a95faee925490896ce6866028cb9612581fad1cc9f88acde7ff02330528"}
Oct 30 19:32:54.925158 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:54.925082    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="332c7a95faee925490896ce6866028cb9612581fad1cc9f88acde7ff02330528"
Oct 30 19:32:58.251732 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:58.251698    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d8ac22a1-xm87l"]
Oct 30 19:32:58.252258 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:58.252245    5907 cpu_manager.go:414] "containerMap: removing container" podUID="8c883612-2ee9-49ed-a69d-d3dce0011061" containerName="revoke-keys"
Oct 30 19:32:58.252258 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:58.252259    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8c883612-2ee9-49ed-a69d-d3dce0011061" containerName="revoke-keys"
Oct 30 19:32:58.252374 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:58.252365    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="8c883612-2ee9-49ed-a69d-d3dce0011061" containerName="revoke-keys"
Oct 30 19:32:58.255972 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:58.255956    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8ac22a1-xm87l"
Oct 30 19:32:58.285728 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:58.285702    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:32:58.285984 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:58.285719    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:32:58.289619 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:58.289590    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d8ac22a1-xm87l"]
Oct 30 19:32:58.375816 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:58.375780    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/57f4366c-c0d7-4b52-88aa-f57069f2f596-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d8ac22a1-xm87l\" (UID: \"57f4366c-c0d7-4b52-88aa-f57069f2f596\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8ac22a1-xm87l"
Oct 30 19:32:58.375990 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:58.375836    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/57f4366c-c0d7-4b52-88aa-f57069f2f596-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d8ac22a1-xm87l\" (UID: \"57f4366c-c0d7-4b52-88aa-f57069f2f596\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8ac22a1-xm87l"
Oct 30 19:32:58.375990 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:58.375973    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckctb\" (UniqueName: \"kubernetes.io/projected/57f4366c-c0d7-4b52-88aa-f57069f2f596-kube-api-access-ckctb\") pod \"maas-api-revoke-sub-d8ac22a1-xm87l\" (UID: \"57f4366c-c0d7-4b52-88aa-f57069f2f596\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8ac22a1-xm87l"
Oct 30 19:32:58.477066 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:58.477026    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/57f4366c-c0d7-4b52-88aa-f57069f2f596-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d8ac22a1-xm87l\" (UID: \"57f4366c-c0d7-4b52-88aa-f57069f2f596\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8ac22a1-xm87l"
Oct 30 19:32:58.477247 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:58.477094    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/57f4366c-c0d7-4b52-88aa-f57069f2f596-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d8ac22a1-xm87l\" (UID: \"57f4366c-c0d7-4b52-88aa-f57069f2f596\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8ac22a1-xm87l"
Oct 30 19:32:58.477247 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:58.477183    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckctb\" (UniqueName: \"kubernetes.io/projected/57f4366c-c0d7-4b52-88aa-f57069f2f596-kube-api-access-ckctb\") pod \"maas-api-revoke-sub-d8ac22a1-xm87l\" (UID: \"57f4366c-c0d7-4b52-88aa-f57069f2f596\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8ac22a1-xm87l"
Oct 30 19:32:58.477801 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:58.477728    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/57f4366c-c0d7-4b52-88aa-f57069f2f596-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d8ac22a1-xm87l\" (UID: \"57f4366c-c0d7-4b52-88aa-f57069f2f596\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8ac22a1-xm87l"
Oct 30 19:32:58.479647 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:58.479621    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/57f4366c-c0d7-4b52-88aa-f57069f2f596-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d8ac22a1-xm87l\" (UID: \"57f4366c-c0d7-4b52-88aa-f57069f2f596\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8ac22a1-xm87l"
Oct 30 19:32:58.499061 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:58.499032    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckctb\" (UniqueName: \"kubernetes.io/projected/57f4366c-c0d7-4b52-88aa-f57069f2f596-kube-api-access-ckctb\") pod \"maas-api-revoke-sub-d8ac22a1-xm87l\" (UID: \"57f4366c-c0d7-4b52-88aa-f57069f2f596\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8ac22a1-xm87l"
Oct 30 19:32:58.569262 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:58.569183    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8ac22a1-xm87l"
Oct 30 19:32:58.874081 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:58.874056    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d8ac22a1-xm87l"]
Oct 30 19:32:58.876068 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:32:58.876036    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57f4366c_c0d7_4b52_88aa_f57069f2f596.slice/crio-af8f091fb0e81196e1643932f1b85a1c8604a81afd6da98fbeea9f1211a0400e WatchSource:0}: Error finding container af8f091fb0e81196e1643932f1b85a1c8604a81afd6da98fbeea9f1211a0400e: Status 404 returned error can't find the container with id af8f091fb0e81196e1643932f1b85a1c8604a81afd6da98fbeea9f1211a0400e
Oct 30 19:32:58.953773 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:58.953729    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8ac22a1-xm87l" event={"ID":"57f4366c-c0d7-4b52-88aa-f57069f2f596","Type":"ContainerStarted","Data":"af8f091fb0e81196e1643932f1b85a1c8604a81afd6da98fbeea9f1211a0400e"}
Oct 30 19:32:59.961856 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:59.961824    5907 generic.go:356] "Generic (PLEG): container finished" podID="57f4366c-c0d7-4b52-88aa-f57069f2f596" containerID="722df4ac79655eb0f8b944ff920041264e09143ab09dfa0ea7cfb0c7b2db8096" exitCode=0
Oct 30 19:32:59.962237 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:32:59.961908    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8ac22a1-xm87l" event={"ID":"57f4366c-c0d7-4b52-88aa-f57069f2f596","Type":"ContainerDied","Data":"722df4ac79655eb0f8b944ff920041264e09143ab09dfa0ea7cfb0c7b2db8096"}
Oct 30 19:33:00.484875 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:00.484838    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"]
Oct 30 19:33:00.489244 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:00.489222    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"
Oct 30 19:33:00.518773 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:00.518744    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-4b4c34-openshift-default-dockercfg-w76ld\""
Oct 30 19:33:00.522207 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:00.522181    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"]
Oct 30 19:33:00.616823 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:00.616787    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/819b9937-3bc3-47ac-9631-b6c4fa30a80d-istio-token\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"
Oct 30 19:33:00.616823 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:00.616828    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/819b9937-3bc3-47ac-9631-b6c4fa30a80d-workload-socket\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"
Oct 30 19:33:00.617066 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:00.616854    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/819b9937-3bc3-47ac-9631-b6c4fa30a80d-istio-podinfo\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"
Oct 30 19:33:00.617066 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:00.617019    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/819b9937-3bc3-47ac-9631-b6c4fa30a80d-istio-envoy\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"
Oct 30 19:33:00.617066 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:00.617053    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f42b7\" (UniqueName: \"kubernetes.io/projected/819b9937-3bc3-47ac-9631-b6c4fa30a80d-kube-api-access-f42b7\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"
Oct 30 19:33:00.617165 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:00.617083    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/819b9937-3bc3-47ac-9631-b6c4fa30a80d-istiod-ca-cert\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"
Oct 30 19:33:00.617198 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:00.617182    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/819b9937-3bc3-47ac-9631-b6c4fa30a80d-istio-data\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"
Oct 30 19:33:00.617261 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:00.617243    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/819b9937-3bc3-47ac-9631-b6c4fa30a80d-credential-socket\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"
Oct 30 19:33:00.617293 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:00.617279    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/819b9937-3bc3-47ac-9631-b6c4fa30a80d-workload-certs\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"
Oct 30 19:33:00.718756 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:00.718726    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/819b9937-3bc3-47ac-9631-b6c4fa30a80d-credential-socket\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"
Oct 30 19:33:00.718966 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:00.718766    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/819b9937-3bc3-47ac-9631-b6c4fa30a80d-workload-certs\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"
Oct 30 19:33:00.718966 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:00.718839    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/819b9937-3bc3-47ac-9631-b6c4fa30a80d-istio-token\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"
Oct 30 19:33:00.718966 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:00.718897    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/819b9937-3bc3-47ac-9631-b6c4fa30a80d-workload-socket\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"
Oct 30 19:33:00.718966 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:00.718930    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/819b9937-3bc3-47ac-9631-b6c4fa30a80d-istio-podinfo\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"
Oct 30 19:33:00.719187 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:00.718991    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/819b9937-3bc3-47ac-9631-b6c4fa30a80d-istio-envoy\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"
Oct 30 19:33:00.719187 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:00.719037    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f42b7\" (UniqueName: \"kubernetes.io/projected/819b9937-3bc3-47ac-9631-b6c4fa30a80d-kube-api-access-f42b7\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"
Oct 30 19:33:00.719187 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:00.719087    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/819b9937-3bc3-47ac-9631-b6c4fa30a80d-istiod-ca-cert\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"
Oct 30 19:33:00.719187 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:00.719114    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/819b9937-3bc3-47ac-9631-b6c4fa30a80d-istio-data\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"
Oct 30 19:33:00.719427 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:00.719217    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/819b9937-3bc3-47ac-9631-b6c4fa30a80d-credential-socket\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"
Oct 30 19:33:00.719427 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:00.719372    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/819b9937-3bc3-47ac-9631-b6c4fa30a80d-workload-socket\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"
Oct 30 19:33:00.719538 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:00.719515    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/819b9937-3bc3-47ac-9631-b6c4fa30a80d-istio-data\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"
Oct 30 19:33:00.719538 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:00.719531    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/819b9937-3bc3-47ac-9631-b6c4fa30a80d-workload-certs\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"
Oct 30 19:33:00.719811 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:00.719793    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/819b9937-3bc3-47ac-9631-b6c4fa30a80d-istiod-ca-cert\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"
Oct 30 19:33:00.721716 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:00.721696    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/819b9937-3bc3-47ac-9631-b6c4fa30a80d-istio-podinfo\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"
Oct 30 19:33:00.721827 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:00.721727    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/819b9937-3bc3-47ac-9631-b6c4fa30a80d-istio-envoy\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"
Oct 30 19:33:00.739648 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:00.739577    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/819b9937-3bc3-47ac-9631-b6c4fa30a80d-istio-token\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"
Oct 30 19:33:00.740192 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:00.740169    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f42b7\" (UniqueName: \"kubernetes.io/projected/819b9937-3bc3-47ac-9631-b6c4fa30a80d-kube-api-access-f42b7\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"
Oct 30 19:33:00.803427 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:00.803395    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"
Oct 30 19:33:01.128083 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:01.128048    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"]
Oct 30 19:33:01.131975 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:01.131942    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:33:01.132053 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:01.132019    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:33:01.132092 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:01.132051    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:33:01.160217 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:01.160189    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8ac22a1-xm87l"
Oct 30 19:33:01.327785 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:01.327751    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/57f4366c-c0d7-4b52-88aa-f57069f2f596-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/57f4366c-c0d7-4b52-88aa-f57069f2f596-maas-api-xks-ca\") pod \"57f4366c-c0d7-4b52-88aa-f57069f2f596\" (UID: \"57f4366c-c0d7-4b52-88aa-f57069f2f596\") "
Oct 30 19:33:01.328005 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:01.327943    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/57f4366c-c0d7-4b52-88aa-f57069f2f596-kube-api-access-ckctb\" (UniqueName: \"kubernetes.io/projected/57f4366c-c0d7-4b52-88aa-f57069f2f596-kube-api-access-ckctb\") pod \"57f4366c-c0d7-4b52-88aa-f57069f2f596\" (UID: \"57f4366c-c0d7-4b52-88aa-f57069f2f596\") "
Oct 30 19:33:01.328068 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:01.328033    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/57f4366c-c0d7-4b52-88aa-f57069f2f596-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/57f4366c-c0d7-4b52-88aa-f57069f2f596-maas-api-openshift-ca\") pod \"57f4366c-c0d7-4b52-88aa-f57069f2f596\" (UID: \"57f4366c-c0d7-4b52-88aa-f57069f2f596\") "
Oct 30 19:33:01.328824 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:01.328797    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57f4366c-c0d7-4b52-88aa-f57069f2f596-maas-api-openshift-ca" pod "57f4366c-c0d7-4b52-88aa-f57069f2f596" (UID: "57f4366c-c0d7-4b52-88aa-f57069f2f596"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:01.334361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:01.334329    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57f4366c-c0d7-4b52-88aa-f57069f2f596-maas-api-xks-ca" pod "57f4366c-c0d7-4b52-88aa-f57069f2f596" (UID: "57f4366c-c0d7-4b52-88aa-f57069f2f596"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:01.357934 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:01.350139    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57f4366c-c0d7-4b52-88aa-f57069f2f596-kube-api-access-ckctb" pod "57f4366c-c0d7-4b52-88aa-f57069f2f596" (UID: "57f4366c-c0d7-4b52-88aa-f57069f2f596"). InnerVolumeSpecName "kube-api-access-ckctb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:01.429845 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:01.429803    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/57f4366c-c0d7-4b52-88aa-f57069f2f596-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:01.430146 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:01.430126    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/57f4366c-c0d7-4b52-88aa-f57069f2f596-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:01.430269 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:01.430257    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ckctb\" (UniqueName: \"kubernetes.io/projected/57f4366c-c0d7-4b52-88aa-f57069f2f596-kube-api-access-ckctb\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:02.002390 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:02.002289    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5" event={"ID":"819b9937-3bc3-47ac-9631-b6c4fa30a80d","Type":"ContainerStarted","Data":"e731427a8cd1043751072da437ef799eb543c3b5d60ac0b5602aa9982a7424a9"}
Oct 30 19:33:02.002390 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:02.002352    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5" event={"ID":"819b9937-3bc3-47ac-9631-b6c4fa30a80d","Type":"ContainerStarted","Data":"65243c1dc07173ab9bf3b4ac429cd2a9e8ba70d45c64122882e09eefb2dd0ca2"}
Oct 30 19:33:02.003822 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:02.003801    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8ac22a1-xm87l"
Oct 30 19:33:02.003974 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:02.003830    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d8ac22a1-xm87l" event={"ID":"57f4366c-c0d7-4b52-88aa-f57069f2f596","Type":"ContainerDied","Data":"af8f091fb0e81196e1643932f1b85a1c8604a81afd6da98fbeea9f1211a0400e"}
Oct 30 19:33:02.003974 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:02.003884    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="af8f091fb0e81196e1643932f1b85a1c8604a81afd6da98fbeea9f1211a0400e"
Oct 30 19:33:02.103279 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:02.103220    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5" podStartSLOduration=2.103203629 podStartE2EDuration="2.103203629s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:33:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:33:02.088620416 +0000 UTC m=+693.931552361" watchObservedRunningTime="2026-10-30 19:33:02.103203629 +0000 UTC m=+693.946135568"
Oct 30 19:33:02.804322 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:02.804281    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"
Oct 30 19:33:02.805757 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:02.805731    5907 patch_prober.go:29] interesting pod/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Oct 30 19:33:02.805894 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:02.805792    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5" podUID="819b9937-3bc3-47ac-9631-b6c4fa30a80d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Oct 30 19:33:03.807975 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:03.804485    5907 patch_prober.go:29] interesting pod/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Oct 30 19:33:03.807975 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:03.804563    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5" podUID="819b9937-3bc3-47ac-9631-b6c4fa30a80d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Oct 30 19:33:04.804402 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:04.804364    5907 patch_prober.go:29] interesting pod/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Oct 30 19:33:04.804584 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:04.804423    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5" podUID="819b9937-3bc3-47ac-9631-b6c4fa30a80d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Oct 30 19:33:04.865733 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:04.865691    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8bbfccd1-k7zbj"]
Oct 30 19:33:04.866539 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:04.866520    5907 cpu_manager.go:414] "containerMap: removing container" podUID="57f4366c-c0d7-4b52-88aa-f57069f2f596" containerName="revoke-keys"
Oct 30 19:33:04.866587 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:04.866541    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="57f4366c-c0d7-4b52-88aa-f57069f2f596" containerName="revoke-keys"
Oct 30 19:33:04.866644 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:04.866625    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="57f4366c-c0d7-4b52-88aa-f57069f2f596" containerName="revoke-keys"
Oct 30 19:33:04.870425 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:04.870406    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8bbfccd1-k7zbj"
Oct 30 19:33:04.892656 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:04.890631    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8bbfccd1-k7zbj"]
Oct 30 19:33:04.900648 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:04.900616    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:33:04.900786 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:04.900767    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:33:04.972558 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:04.972522    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c6a39cdf-82d1-44a3-902f-745528b5313b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8bbfccd1-k7zbj\" (UID: \"c6a39cdf-82d1-44a3-902f-745528b5313b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8bbfccd1-k7zbj"
Oct 30 19:33:04.972735 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:04.972582    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjl7d\" (UniqueName: \"kubernetes.io/projected/c6a39cdf-82d1-44a3-902f-745528b5313b-kube-api-access-gjl7d\") pod \"maas-api-revoke-sub-8bbfccd1-k7zbj\" (UID: \"c6a39cdf-82d1-44a3-902f-745528b5313b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8bbfccd1-k7zbj"
Oct 30 19:33:04.972735 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:04.972695    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c6a39cdf-82d1-44a3-902f-745528b5313b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8bbfccd1-k7zbj\" (UID: \"c6a39cdf-82d1-44a3-902f-745528b5313b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8bbfccd1-k7zbj"
Oct 30 19:33:05.074216 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:05.074128    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c6a39cdf-82d1-44a3-902f-745528b5313b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8bbfccd1-k7zbj\" (UID: \"c6a39cdf-82d1-44a3-902f-745528b5313b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8bbfccd1-k7zbj"
Oct 30 19:33:05.074216 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:05.074172    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjl7d\" (UniqueName: \"kubernetes.io/projected/c6a39cdf-82d1-44a3-902f-745528b5313b-kube-api-access-gjl7d\") pod \"maas-api-revoke-sub-8bbfccd1-k7zbj\" (UID: \"c6a39cdf-82d1-44a3-902f-745528b5313b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8bbfccd1-k7zbj"
Oct 30 19:33:05.074447 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:05.074247    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c6a39cdf-82d1-44a3-902f-745528b5313b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8bbfccd1-k7zbj\" (UID: \"c6a39cdf-82d1-44a3-902f-745528b5313b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8bbfccd1-k7zbj"
Oct 30 19:33:05.074845 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:05.074817    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c6a39cdf-82d1-44a3-902f-745528b5313b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8bbfccd1-k7zbj\" (UID: \"c6a39cdf-82d1-44a3-902f-745528b5313b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8bbfccd1-k7zbj"
Oct 30 19:33:05.076802 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:05.076780    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c6a39cdf-82d1-44a3-902f-745528b5313b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8bbfccd1-k7zbj\" (UID: \"c6a39cdf-82d1-44a3-902f-745528b5313b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8bbfccd1-k7zbj"
Oct 30 19:33:05.098428 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:05.098402    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjl7d\" (UniqueName: \"kubernetes.io/projected/c6a39cdf-82d1-44a3-902f-745528b5313b-kube-api-access-gjl7d\") pod \"maas-api-revoke-sub-8bbfccd1-k7zbj\" (UID: \"c6a39cdf-82d1-44a3-902f-745528b5313b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8bbfccd1-k7zbj"
Oct 30 19:33:05.186341 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:05.186304    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8bbfccd1-k7zbj"
Oct 30 19:33:05.527875 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:05.527824    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8bbfccd1-k7zbj"]
Oct 30 19:33:05.539576 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:33:05.539529    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6a39cdf_82d1_44a3_902f_745528b5313b.slice/crio-a9eb341ee455f320af55e92f75b00b400b4a4e4880e6e520e061c52bb9bd7f40 WatchSource:0}: Error finding container a9eb341ee455f320af55e92f75b00b400b4a4e4880e6e520e061c52bb9bd7f40: Status 404 returned error can't find the container with id a9eb341ee455f320af55e92f75b00b400b4a4e4880e6e520e061c52bb9bd7f40
Oct 30 19:33:05.804586 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:05.804505    5907 patch_prober.go:29] interesting pod/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Oct 30 19:33:05.804737 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:05.804575    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5" podUID="819b9937-3bc3-47ac-9631-b6c4fa30a80d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Oct 30 19:33:06.049238 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:06.049205    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8bbfccd1-k7zbj" event={"ID":"c6a39cdf-82d1-44a3-902f-745528b5313b","Type":"ContainerStarted","Data":"3835a91f398e61112d95a6c3b8d902409a01afeacd203f95cf158072da0dec40"}
Oct 30 19:33:06.049676 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:06.049288    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8bbfccd1-k7zbj" event={"ID":"c6a39cdf-82d1-44a3-902f-745528b5313b","Type":"ContainerStarted","Data":"a9eb341ee455f320af55e92f75b00b400b4a4e4880e6e520e061c52bb9bd7f40"}
Oct 30 19:33:06.091034 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:06.090917    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8bbfccd1-k7zbj" podStartSLOduration=2.090887505 podStartE2EDuration="2.090887505s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:33:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:33:06.089980863 +0000 UTC m=+697.932912805" watchObservedRunningTime="2026-10-30 19:33:06.090887505 +0000 UTC m=+697.933819451"
Oct 30 19:33:06.803825 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:06.803790    5907 patch_prober.go:29] interesting pod/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Oct 30 19:33:06.804119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:06.803854    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5" podUID="819b9937-3bc3-47ac-9631-b6c4fa30a80d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Oct 30 19:33:07.057470 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:07.057442    5907 generic.go:356] "Generic (PLEG): container finished" podID="c6a39cdf-82d1-44a3-902f-745528b5313b" containerID="3835a91f398e61112d95a6c3b8d902409a01afeacd203f95cf158072da0dec40" exitCode=0
Oct 30 19:33:07.057948 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:07.057632    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8bbfccd1-k7zbj" event={"ID":"c6a39cdf-82d1-44a3-902f-745528b5313b","Type":"ContainerDied","Data":"3835a91f398e61112d95a6c3b8d902409a01afeacd203f95cf158072da0dec40"}
Oct 30 19:33:07.973943 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:07.973895    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"
Oct 30 19:33:07.974248 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:07.974225    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"
Oct 30 19:33:07.975358 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:07.975334    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"
Oct 30 19:33:08.257204 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:08.257179    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8bbfccd1-k7zbj"
Oct 30 19:33:08.423228 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:08.423163    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c6a39cdf-82d1-44a3-902f-745528b5313b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c6a39cdf-82d1-44a3-902f-745528b5313b-maas-api-xks-ca\") pod \"c6a39cdf-82d1-44a3-902f-745528b5313b\" (UID: \"c6a39cdf-82d1-44a3-902f-745528b5313b\") "
Oct 30 19:33:08.423403 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:08.423282    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c6a39cdf-82d1-44a3-902f-745528b5313b-kube-api-access-gjl7d\" (UniqueName: \"kubernetes.io/projected/c6a39cdf-82d1-44a3-902f-745528b5313b-kube-api-access-gjl7d\") pod \"c6a39cdf-82d1-44a3-902f-745528b5313b\" (UID: \"c6a39cdf-82d1-44a3-902f-745528b5313b\") "
Oct 30 19:33:08.423403 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:08.423386    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c6a39cdf-82d1-44a3-902f-745528b5313b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c6a39cdf-82d1-44a3-902f-745528b5313b-maas-api-openshift-ca\") pod \"c6a39cdf-82d1-44a3-902f-745528b5313b\" (UID: \"c6a39cdf-82d1-44a3-902f-745528b5313b\") "
Oct 30 19:33:08.424076 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:08.424047    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6a39cdf-82d1-44a3-902f-745528b5313b-maas-api-openshift-ca" pod "c6a39cdf-82d1-44a3-902f-745528b5313b" (UID: "c6a39cdf-82d1-44a3-902f-745528b5313b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:08.425454 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:08.425427    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6a39cdf-82d1-44a3-902f-745528b5313b-maas-api-xks-ca" pod "c6a39cdf-82d1-44a3-902f-745528b5313b" (UID: "c6a39cdf-82d1-44a3-902f-745528b5313b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:08.428354 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:08.428324    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6a39cdf-82d1-44a3-902f-745528b5313b-kube-api-access-gjl7d" pod "c6a39cdf-82d1-44a3-902f-745528b5313b" (UID: "c6a39cdf-82d1-44a3-902f-745528b5313b"). InnerVolumeSpecName "kube-api-access-gjl7d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:08.525324 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:08.525236    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c6a39cdf-82d1-44a3-902f-745528b5313b-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:08.525552 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:08.525526    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjl7d\" (UniqueName: \"kubernetes.io/projected/c6a39cdf-82d1-44a3-902f-745528b5313b-kube-api-access-gjl7d\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:08.525552 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:08.525550    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c6a39cdf-82d1-44a3-902f-745528b5313b-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:09.071581 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:09.071552    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8bbfccd1-k7zbj"
Oct 30 19:33:09.071731 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:09.071587    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8bbfccd1-k7zbj" event={"ID":"c6a39cdf-82d1-44a3-902f-745528b5313b","Type":"ContainerDied","Data":"a9eb341ee455f320af55e92f75b00b400b4a4e4880e6e520e061c52bb9bd7f40"}
Oct 30 19:33:09.071731 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:09.071636    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a9eb341ee455f320af55e92f75b00b400b4a4e4880e6e520e061c52bb9bd7f40"
Oct 30 19:33:12.620296 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:12.620261    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d23bf141-d6lxg"]
Oct 30 19:33:12.621070 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:12.621048    5907 cpu_manager.go:414] "containerMap: removing container" podUID="c6a39cdf-82d1-44a3-902f-745528b5313b" containerName="revoke-keys"
Oct 30 19:33:12.621070 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:12.621069    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c6a39cdf-82d1-44a3-902f-745528b5313b" containerName="revoke-keys"
Oct 30 19:33:12.621272 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:12.621258    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="c6a39cdf-82d1-44a3-902f-745528b5313b" containerName="revoke-keys"
Oct 30 19:33:12.624964 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:12.624945    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d23bf141-d6lxg"
Oct 30 19:33:12.650241 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:12.650209    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d23bf141-d6lxg"]
Oct 30 19:33:12.657360 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:12.657321    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:33:12.657360 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:12.657350    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:33:12.777750 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:12.777715    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wr2hk\" (UniqueName: \"kubernetes.io/projected/4c779354-fa02-4c26-999a-bd75c8a3e351-kube-api-access-wr2hk\") pod \"maas-api-revoke-sub-d23bf141-d6lxg\" (UID: \"4c779354-fa02-4c26-999a-bd75c8a3e351\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d23bf141-d6lxg"
Oct 30 19:33:12.777994 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:12.777971    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4c779354-fa02-4c26-999a-bd75c8a3e351-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d23bf141-d6lxg\" (UID: \"4c779354-fa02-4c26-999a-bd75c8a3e351\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d23bf141-d6lxg"
Oct 30 19:33:12.778058 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:12.778004    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4c779354-fa02-4c26-999a-bd75c8a3e351-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d23bf141-d6lxg\" (UID: \"4c779354-fa02-4c26-999a-bd75c8a3e351\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d23bf141-d6lxg"
Oct 30 19:33:12.878597 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:12.878512    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4c779354-fa02-4c26-999a-bd75c8a3e351-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d23bf141-d6lxg\" (UID: \"4c779354-fa02-4c26-999a-bd75c8a3e351\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d23bf141-d6lxg"
Oct 30 19:33:12.878597 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:12.878547    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4c779354-fa02-4c26-999a-bd75c8a3e351-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d23bf141-d6lxg\" (UID: \"4c779354-fa02-4c26-999a-bd75c8a3e351\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d23bf141-d6lxg"
Oct 30 19:33:12.878597 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:12.878590    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wr2hk\" (UniqueName: \"kubernetes.io/projected/4c779354-fa02-4c26-999a-bd75c8a3e351-kube-api-access-wr2hk\") pod \"maas-api-revoke-sub-d23bf141-d6lxg\" (UID: \"4c779354-fa02-4c26-999a-bd75c8a3e351\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d23bf141-d6lxg"
Oct 30 19:33:12.879222 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:12.879203    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4c779354-fa02-4c26-999a-bd75c8a3e351-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d23bf141-d6lxg\" (UID: \"4c779354-fa02-4c26-999a-bd75c8a3e351\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d23bf141-d6lxg"
Oct 30 19:33:12.881091 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:12.881067    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4c779354-fa02-4c26-999a-bd75c8a3e351-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d23bf141-d6lxg\" (UID: \"4c779354-fa02-4c26-999a-bd75c8a3e351\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d23bf141-d6lxg"
Oct 30 19:33:12.900056 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:12.900020    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wr2hk\" (UniqueName: \"kubernetes.io/projected/4c779354-fa02-4c26-999a-bd75c8a3e351-kube-api-access-wr2hk\") pod \"maas-api-revoke-sub-d23bf141-d6lxg\" (UID: \"4c779354-fa02-4c26-999a-bd75c8a3e351\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d23bf141-d6lxg"
Oct 30 19:33:12.938743 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:12.938711    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d23bf141-d6lxg"
Oct 30 19:33:13.249595 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:13.249562    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d23bf141-d6lxg"]
Oct 30 19:33:13.251620 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:33:13.251593    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c779354_fa02_4c26_999a_bd75c8a3e351.slice/crio-704848758b8d45d04184632b11f00d2c079cbe73b0fb47fa70a450a419494f38 WatchSource:0}: Error finding container 704848758b8d45d04184632b11f00d2c079cbe73b0fb47fa70a450a419494f38: Status 404 returned error can't find the container with id 704848758b8d45d04184632b11f00d2c079cbe73b0fb47fa70a450a419494f38
Oct 30 19:33:13.319034 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:13.318996    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a566702d-jznrx"]
Oct 30 19:33:13.324554 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:13.324528    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a566702d-jznrx"
Oct 30 19:33:13.344360 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:13.344328    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a566702d-jznrx"]
Oct 30 19:33:13.487424 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:13.487385    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6d1935cb-a979-4aad-aa3f-988c94abd3d6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a566702d-jznrx\" (UID: \"6d1935cb-a979-4aad-aa3f-988c94abd3d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a566702d-jznrx"
Oct 30 19:33:13.487592 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:13.487510    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6d1935cb-a979-4aad-aa3f-988c94abd3d6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a566702d-jznrx\" (UID: \"6d1935cb-a979-4aad-aa3f-988c94abd3d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a566702d-jznrx"
Oct 30 19:33:13.487592 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:13.487540    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cstnq\" (UniqueName: \"kubernetes.io/projected/6d1935cb-a979-4aad-aa3f-988c94abd3d6-kube-api-access-cstnq\") pod \"maas-api-revoke-sub-a566702d-jznrx\" (UID: \"6d1935cb-a979-4aad-aa3f-988c94abd3d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a566702d-jznrx"
Oct 30 19:33:13.597769 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:13.589214    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6d1935cb-a979-4aad-aa3f-988c94abd3d6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a566702d-jznrx\" (UID: \"6d1935cb-a979-4aad-aa3f-988c94abd3d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a566702d-jznrx"
Oct 30 19:33:13.597769 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:13.589323    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6d1935cb-a979-4aad-aa3f-988c94abd3d6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a566702d-jznrx\" (UID: \"6d1935cb-a979-4aad-aa3f-988c94abd3d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a566702d-jznrx"
Oct 30 19:33:13.597769 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:13.589361    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cstnq\" (UniqueName: \"kubernetes.io/projected/6d1935cb-a979-4aad-aa3f-988c94abd3d6-kube-api-access-cstnq\") pod \"maas-api-revoke-sub-a566702d-jznrx\" (UID: \"6d1935cb-a979-4aad-aa3f-988c94abd3d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a566702d-jznrx"
Oct 30 19:33:13.597769 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:13.590574    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6d1935cb-a979-4aad-aa3f-988c94abd3d6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a566702d-jznrx\" (UID: \"6d1935cb-a979-4aad-aa3f-988c94abd3d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a566702d-jznrx"
Oct 30 19:33:13.599511 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:13.598724    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6d1935cb-a979-4aad-aa3f-988c94abd3d6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a566702d-jznrx\" (UID: \"6d1935cb-a979-4aad-aa3f-988c94abd3d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a566702d-jznrx"
Oct 30 19:33:13.610879 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:13.610835    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cstnq\" (UniqueName: \"kubernetes.io/projected/6d1935cb-a979-4aad-aa3f-988c94abd3d6-kube-api-access-cstnq\") pod \"maas-api-revoke-sub-a566702d-jznrx\" (UID: \"6d1935cb-a979-4aad-aa3f-988c94abd3d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a566702d-jznrx"
Oct 30 19:33:13.638852 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:13.638822    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a566702d-jznrx"
Oct 30 19:33:13.943572 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:13.943542    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a566702d-jznrx"]
Oct 30 19:33:13.945299 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:33:13.945264    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d1935cb_a979_4aad_aa3f_988c94abd3d6.slice/crio-b477c67bfe4ba3eac263d0733c9a53be17d595a21ad9d7ce52b888b282b500cc WatchSource:0}: Error finding container b477c67bfe4ba3eac263d0733c9a53be17d595a21ad9d7ce52b888b282b500cc: Status 404 returned error can't find the container with id b477c67bfe4ba3eac263d0733c9a53be17d595a21ad9d7ce52b888b282b500cc
Oct 30 19:33:13.984925 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:13.984903    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f74b9672-t56vz"]
Oct 30 19:33:13.990566 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:13.990546    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f74b9672-t56vz"
Oct 30 19:33:14.011370 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.011124    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f74b9672-t56vz"]
Oct 30 19:33:14.096161 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.096127    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d4da88bf-a2b6-4055-8d91-50f0d10decdd-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f74b9672-t56vz\" (UID: \"d4da88bf-a2b6-4055-8d91-50f0d10decdd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f74b9672-t56vz"
Oct 30 19:33:14.096161 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.096167    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d4da88bf-a2b6-4055-8d91-50f0d10decdd-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f74b9672-t56vz\" (UID: \"d4da88bf-a2b6-4055-8d91-50f0d10decdd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f74b9672-t56vz"
Oct 30 19:33:14.096390 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.096190    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhgz8\" (UniqueName: \"kubernetes.io/projected/d4da88bf-a2b6-4055-8d91-50f0d10decdd-kube-api-access-jhgz8\") pod \"maas-api-revoke-sub-f74b9672-t56vz\" (UID: \"d4da88bf-a2b6-4055-8d91-50f0d10decdd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f74b9672-t56vz"
Oct 30 19:33:14.104240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.104202    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d23bf141-d6lxg" event={"ID":"4c779354-fa02-4c26-999a-bd75c8a3e351","Type":"ContainerStarted","Data":"c31f301ba02082d628a0a9c39ff158c5ebe5f553db5f4846aabe8e71d59a0bd2"}
Oct 30 19:33:14.104400 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.104269    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d23bf141-d6lxg" event={"ID":"4c779354-fa02-4c26-999a-bd75c8a3e351","Type":"ContainerStarted","Data":"704848758b8d45d04184632b11f00d2c079cbe73b0fb47fa70a450a419494f38"}
Oct 30 19:33:14.105549 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.105525    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a566702d-jznrx" event={"ID":"6d1935cb-a979-4aad-aa3f-988c94abd3d6","Type":"ContainerStarted","Data":"b477c67bfe4ba3eac263d0733c9a53be17d595a21ad9d7ce52b888b282b500cc"}
Oct 30 19:33:14.148251 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.148206    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d23bf141-d6lxg" podStartSLOduration=2.148191959 podStartE2EDuration="2.148191959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:33:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:33:14.14533064 +0000 UTC m=+705.988262579" watchObservedRunningTime="2026-10-30 19:33:14.148191959 +0000 UTC m=+705.991123914"
Oct 30 19:33:14.197523 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.197440    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d4da88bf-a2b6-4055-8d91-50f0d10decdd-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f74b9672-t56vz\" (UID: \"d4da88bf-a2b6-4055-8d91-50f0d10decdd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f74b9672-t56vz"
Oct 30 19:33:14.197523 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.197477    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d4da88bf-a2b6-4055-8d91-50f0d10decdd-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f74b9672-t56vz\" (UID: \"d4da88bf-a2b6-4055-8d91-50f0d10decdd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f74b9672-t56vz"
Oct 30 19:33:14.197523 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.197496    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhgz8\" (UniqueName: \"kubernetes.io/projected/d4da88bf-a2b6-4055-8d91-50f0d10decdd-kube-api-access-jhgz8\") pod \"maas-api-revoke-sub-f74b9672-t56vz\" (UID: \"d4da88bf-a2b6-4055-8d91-50f0d10decdd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f74b9672-t56vz"
Oct 30 19:33:14.198210 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.198192    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d4da88bf-a2b6-4055-8d91-50f0d10decdd-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f74b9672-t56vz\" (UID: \"d4da88bf-a2b6-4055-8d91-50f0d10decdd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f74b9672-t56vz"
Oct 30 19:33:14.200088 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.200068    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d4da88bf-a2b6-4055-8d91-50f0d10decdd-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f74b9672-t56vz\" (UID: \"d4da88bf-a2b6-4055-8d91-50f0d10decdd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f74b9672-t56vz"
Oct 30 19:33:14.219006 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.218982    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhgz8\" (UniqueName: \"kubernetes.io/projected/d4da88bf-a2b6-4055-8d91-50f0d10decdd-kube-api-access-jhgz8\") pod \"maas-api-revoke-sub-f74b9672-t56vz\" (UID: \"d4da88bf-a2b6-4055-8d91-50f0d10decdd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f74b9672-t56vz"
Oct 30 19:33:14.305994 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.305964    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f74b9672-t56vz"
Oct 30 19:33:14.610964 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.610940    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f74b9672-t56vz"]
Oct 30 19:33:14.613191 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:33:14.613161    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4da88bf_a2b6_4055_8d91_50f0d10decdd.slice/crio-d69a4c8cca4439640857690699589681cd36e0dd6fe654d3688cbed457b7d466 WatchSource:0}: Error finding container d69a4c8cca4439640857690699589681cd36e0dd6fe654d3688cbed457b7d466: Status 404 returned error can't find the container with id d69a4c8cca4439640857690699589681cd36e0dd6fe654d3688cbed457b7d466
Oct 30 19:33:14.698132 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.698097    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"]
Oct 30 19:33:14.703683 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.703661    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"
Oct 30 19:33:14.740687 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.740640    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-544fc8-openshift-default-dockercfg-t64zf\""
Oct 30 19:33:14.743959 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.743934    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"]
Oct 30 19:33:14.804824 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.804791    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e440f6c0-de4c-4a55-af89-7ea0552419a9-workload-socket\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"
Oct 30 19:33:14.804988 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.804847    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zmlf\" (UniqueName: \"kubernetes.io/projected/e440f6c0-de4c-4a55-af89-7ea0552419a9-kube-api-access-7zmlf\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"
Oct 30 19:33:14.804988 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.804927    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e440f6c0-de4c-4a55-af89-7ea0552419a9-istio-envoy\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"
Oct 30 19:33:14.804988 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.804969    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e440f6c0-de4c-4a55-af89-7ea0552419a9-workload-certs\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"
Oct 30 19:33:14.805108 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.805053    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e440f6c0-de4c-4a55-af89-7ea0552419a9-istio-token\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"
Oct 30 19:33:14.805108 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.805086    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e440f6c0-de4c-4a55-af89-7ea0552419a9-istiod-ca-cert\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"
Oct 30 19:33:14.805200 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.805180    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e440f6c0-de4c-4a55-af89-7ea0552419a9-credential-socket\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"
Oct 30 19:33:14.805241 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.805221    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e440f6c0-de4c-4a55-af89-7ea0552419a9-istio-data\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"
Oct 30 19:33:14.805303 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.805275    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e440f6c0-de4c-4a55-af89-7ea0552419a9-istio-podinfo\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"
Oct 30 19:33:14.906549 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.906453    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zmlf\" (UniqueName: \"kubernetes.io/projected/e440f6c0-de4c-4a55-af89-7ea0552419a9-kube-api-access-7zmlf\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"
Oct 30 19:33:14.906549 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.906515    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e440f6c0-de4c-4a55-af89-7ea0552419a9-istio-envoy\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"
Oct 30 19:33:14.906753 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.906560    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e440f6c0-de4c-4a55-af89-7ea0552419a9-workload-certs\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"
Oct 30 19:33:14.906753 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.906696    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e440f6c0-de4c-4a55-af89-7ea0552419a9-istio-token\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"
Oct 30 19:33:14.906753 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.906727    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e440f6c0-de4c-4a55-af89-7ea0552419a9-istiod-ca-cert\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"
Oct 30 19:33:14.906937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.906805    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e440f6c0-de4c-4a55-af89-7ea0552419a9-credential-socket\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"
Oct 30 19:33:14.906937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.906851    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e440f6c0-de4c-4a55-af89-7ea0552419a9-istio-data\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"
Oct 30 19:33:14.906937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.906916    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e440f6c0-de4c-4a55-af89-7ea0552419a9-istio-podinfo\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"
Oct 30 19:33:14.907072 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.906969    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e440f6c0-de4c-4a55-af89-7ea0552419a9-workload-certs\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"
Oct 30 19:33:14.907072 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.907036    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e440f6c0-de4c-4a55-af89-7ea0552419a9-workload-socket\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"
Oct 30 19:33:14.907204 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.907178    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e440f6c0-de4c-4a55-af89-7ea0552419a9-credential-socket\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"
Oct 30 19:33:14.907290 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.907264    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e440f6c0-de4c-4a55-af89-7ea0552419a9-istio-data\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"
Oct 30 19:33:14.907400 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.907381    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e440f6c0-de4c-4a55-af89-7ea0552419a9-workload-socket\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"
Oct 30 19:33:14.907518 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.907496    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e440f6c0-de4c-4a55-af89-7ea0552419a9-istiod-ca-cert\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"
Oct 30 19:33:14.910278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.910253    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e440f6c0-de4c-4a55-af89-7ea0552419a9-istio-envoy\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"
Oct 30 19:33:14.910400 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.910384    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e440f6c0-de4c-4a55-af89-7ea0552419a9-istio-podinfo\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"
Oct 30 19:33:14.927689 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.927665    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e440f6c0-de4c-4a55-af89-7ea0552419a9-istio-token\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"
Oct 30 19:33:14.927810 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:14.927736    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zmlf\" (UniqueName: \"kubernetes.io/projected/e440f6c0-de4c-4a55-af89-7ea0552419a9-kube-api-access-7zmlf\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"
Oct 30 19:33:15.025347 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:15.025319    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"
Oct 30 19:33:15.132990 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:15.132953    5907 generic.go:356] "Generic (PLEG): container finished" podID="6d1935cb-a979-4aad-aa3f-988c94abd3d6" containerID="1a43838a9b9076a089c568c2c558d5c400bd5c869905bf242827c7f3f4acb2c6" exitCode=0
Oct 30 19:33:15.133212 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:15.133170    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a566702d-jznrx" event={"ID":"6d1935cb-a979-4aad-aa3f-988c94abd3d6","Type":"ContainerDied","Data":"1a43838a9b9076a089c568c2c558d5c400bd5c869905bf242827c7f3f4acb2c6"}
Oct 30 19:33:15.136005 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:15.135973    5907 generic.go:356] "Generic (PLEG): container finished" podID="4c779354-fa02-4c26-999a-bd75c8a3e351" containerID="c31f301ba02082d628a0a9c39ff158c5ebe5f553db5f4846aabe8e71d59a0bd2" exitCode=0
Oct 30 19:33:15.136159 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:15.136068    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d23bf141-d6lxg" event={"ID":"4c779354-fa02-4c26-999a-bd75c8a3e351","Type":"ContainerDied","Data":"c31f301ba02082d628a0a9c39ff158c5ebe5f553db5f4846aabe8e71d59a0bd2"}
Oct 30 19:33:15.139151 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:15.139115    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f74b9672-t56vz" event={"ID":"d4da88bf-a2b6-4055-8d91-50f0d10decdd","Type":"ContainerStarted","Data":"1b34151b154231ecd8b9c2d1e4d685eb2cd68ad0c027d1de9a0fbab932f5f2b7"}
Oct 30 19:33:15.139324 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:15.139169    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f74b9672-t56vz" event={"ID":"d4da88bf-a2b6-4055-8d91-50f0d10decdd","Type":"ContainerStarted","Data":"d69a4c8cca4439640857690699589681cd36e0dd6fe654d3688cbed457b7d466"}
Oct 30 19:33:15.220028 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:15.219919    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f74b9672-t56vz" podStartSLOduration=2.219901193 podStartE2EDuration="2.219901193s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:33:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:33:15.218753284 +0000 UTC m=+707.061685220" watchObservedRunningTime="2026-10-30 19:33:15.219901193 +0000 UTC m=+707.062833132"
Oct 30 19:33:15.381257 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:15.381218    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"]
Oct 30 19:33:15.382274 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:33:15.382250    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode440f6c0_de4c_4a55_af89_7ea0552419a9.slice/crio-21e27caa1a1ddb62926ad77027416d38f21f8cebb9be07079f95996a6c160847 WatchSource:0}: Error finding container 21e27caa1a1ddb62926ad77027416d38f21f8cebb9be07079f95996a6c160847: Status 404 returned error can't find the container with id 21e27caa1a1ddb62926ad77027416d38f21f8cebb9be07079f95996a6c160847
Oct 30 19:33:15.384688 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:15.384658    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:33:15.384767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:15.384717    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:33:15.384767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:15.384750    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:33:15.995701 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:15.995611    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:33:16.146229 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:16.146191    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc" event={"ID":"e440f6c0-de4c-4a55-af89-7ea0552419a9","Type":"ContainerStarted","Data":"22f20b47048fa05eb29274c8c9834003bc3152b321c440cbcfb3da19a574c460"}
Oct 30 19:33:16.146435 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:16.146251    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc" event={"ID":"e440f6c0-de4c-4a55-af89-7ea0552419a9","Type":"ContainerStarted","Data":"21e27caa1a1ddb62926ad77027416d38f21f8cebb9be07079f95996a6c160847"}
Oct 30 19:33:16.147985 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:16.147953    5907 generic.go:356] "Generic (PLEG): container finished" podID="d4da88bf-a2b6-4055-8d91-50f0d10decdd" containerID="1b34151b154231ecd8b9c2d1e4d685eb2cd68ad0c027d1de9a0fbab932f5f2b7" exitCode=0
Oct 30 19:33:16.148108 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:16.148020    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f74b9672-t56vz" event={"ID":"d4da88bf-a2b6-4055-8d91-50f0d10decdd","Type":"ContainerDied","Data":"1b34151b154231ecd8b9c2d1e4d685eb2cd68ad0c027d1de9a0fbab932f5f2b7"}
Oct 30 19:33:16.198733 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:16.198660    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc" podStartSLOduration=2.198642468 podStartE2EDuration="2.198642468s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:33:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:33:16.194529791 +0000 UTC m=+708.037461729" watchObservedRunningTime="2026-10-30 19:33:16.198642468 +0000 UTC m=+708.041574406"
Oct 30 19:33:16.354031 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:16.353999    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d23bf141-d6lxg"
Oct 30 19:33:16.439585 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:16.439272    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a566702d-jznrx"
Oct 30 19:33:16.555275 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:16.555185    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4c779354-fa02-4c26-999a-bd75c8a3e351-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4c779354-fa02-4c26-999a-bd75c8a3e351-maas-api-xks-ca\") pod \"4c779354-fa02-4c26-999a-bd75c8a3e351\" (UID: \"4c779354-fa02-4c26-999a-bd75c8a3e351\") "
Oct 30 19:33:16.555275 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:16.555226    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6d1935cb-a979-4aad-aa3f-988c94abd3d6-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6d1935cb-a979-4aad-aa3f-988c94abd3d6-maas-api-xks-ca\") pod \"6d1935cb-a979-4aad-aa3f-988c94abd3d6\" (UID: \"6d1935cb-a979-4aad-aa3f-988c94abd3d6\") "
Oct 30 19:33:16.555275 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:16.555270    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4c779354-fa02-4c26-999a-bd75c8a3e351-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4c779354-fa02-4c26-999a-bd75c8a3e351-maas-api-openshift-ca\") pod \"4c779354-fa02-4c26-999a-bd75c8a3e351\" (UID: \"4c779354-fa02-4c26-999a-bd75c8a3e351\") "
Oct 30 19:33:16.555495 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:16.555330    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6d1935cb-a979-4aad-aa3f-988c94abd3d6-kube-api-access-cstnq\" (UniqueName: \"kubernetes.io/projected/6d1935cb-a979-4aad-aa3f-988c94abd3d6-kube-api-access-cstnq\") pod \"6d1935cb-a979-4aad-aa3f-988c94abd3d6\" (UID: \"6d1935cb-a979-4aad-aa3f-988c94abd3d6\") "
Oct 30 19:33:16.555495 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:16.555369    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6d1935cb-a979-4aad-aa3f-988c94abd3d6-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6d1935cb-a979-4aad-aa3f-988c94abd3d6-maas-api-openshift-ca\") pod \"6d1935cb-a979-4aad-aa3f-988c94abd3d6\" (UID: \"6d1935cb-a979-4aad-aa3f-988c94abd3d6\") "
Oct 30 19:33:16.555495 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:16.555430    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4c779354-fa02-4c26-999a-bd75c8a3e351-kube-api-access-wr2hk\" (UniqueName: \"kubernetes.io/projected/4c779354-fa02-4c26-999a-bd75c8a3e351-kube-api-access-wr2hk\") pod \"4c779354-fa02-4c26-999a-bd75c8a3e351\" (UID: \"4c779354-fa02-4c26-999a-bd75c8a3e351\") "
Oct 30 19:33:16.555716 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:16.555679    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c779354-fa02-4c26-999a-bd75c8a3e351-maas-api-openshift-ca" pod "4c779354-fa02-4c26-999a-bd75c8a3e351" (UID: "4c779354-fa02-4c26-999a-bd75c8a3e351"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:16.555896 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:16.555790    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d1935cb-a979-4aad-aa3f-988c94abd3d6-maas-api-openshift-ca" pod "6d1935cb-a979-4aad-aa3f-988c94abd3d6" (UID: "6d1935cb-a979-4aad-aa3f-988c94abd3d6"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:16.556031 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:16.556011    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6d1935cb-a979-4aad-aa3f-988c94abd3d6-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:16.556091 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:16.556037    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4c779354-fa02-4c26-999a-bd75c8a3e351-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:16.557793 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:16.557766    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d1935cb-a979-4aad-aa3f-988c94abd3d6-kube-api-access-cstnq" pod "6d1935cb-a979-4aad-aa3f-988c94abd3d6" (UID: "6d1935cb-a979-4aad-aa3f-988c94abd3d6"). InnerVolumeSpecName "kube-api-access-cstnq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:16.558100 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:16.558075    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d1935cb-a979-4aad-aa3f-988c94abd3d6-maas-api-xks-ca" pod "6d1935cb-a979-4aad-aa3f-988c94abd3d6" (UID: "6d1935cb-a979-4aad-aa3f-988c94abd3d6"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:16.558215 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:16.558191    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c779354-fa02-4c26-999a-bd75c8a3e351-maas-api-xks-ca" pod "4c779354-fa02-4c26-999a-bd75c8a3e351" (UID: "4c779354-fa02-4c26-999a-bd75c8a3e351"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:16.558536 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:16.558514    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c779354-fa02-4c26-999a-bd75c8a3e351-kube-api-access-wr2hk" pod "4c779354-fa02-4c26-999a-bd75c8a3e351" (UID: "4c779354-fa02-4c26-999a-bd75c8a3e351"). InnerVolumeSpecName "kube-api-access-wr2hk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:16.656753 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:16.656719    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wr2hk\" (UniqueName: \"kubernetes.io/projected/4c779354-fa02-4c26-999a-bd75c8a3e351-kube-api-access-wr2hk\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:16.656753 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:16.656749    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4c779354-fa02-4c26-999a-bd75c8a3e351-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:16.656753 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:16.656758    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6d1935cb-a979-4aad-aa3f-988c94abd3d6-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:16.657071 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:16.656771    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cstnq\" (UniqueName: \"kubernetes.io/projected/6d1935cb-a979-4aad-aa3f-988c94abd3d6-kube-api-access-cstnq\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:17.039888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:17.034617    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"
Oct 30 19:33:17.043230 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:17.041988    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 19:33:17.043230 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:17.042053    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc" podUID="e440f6c0-de4c-4a55-af89-7ea0552419a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 19:33:17.157000 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:17.156969    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d23bf141-d6lxg"
Oct 30 19:33:17.157214 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:17.156994    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d23bf141-d6lxg" event={"ID":"4c779354-fa02-4c26-999a-bd75c8a3e351","Type":"ContainerDied","Data":"704848758b8d45d04184632b11f00d2c079cbe73b0fb47fa70a450a419494f38"}
Oct 30 19:33:17.157214 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:17.157054    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="704848758b8d45d04184632b11f00d2c079cbe73b0fb47fa70a450a419494f38"
Oct 30 19:33:17.158695 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:17.158675    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a566702d-jznrx"
Oct 30 19:33:17.162736 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:17.161995    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a566702d-jznrx" event={"ID":"6d1935cb-a979-4aad-aa3f-988c94abd3d6","Type":"ContainerDied","Data":"b477c67bfe4ba3eac263d0733c9a53be17d595a21ad9d7ce52b888b282b500cc"}
Oct 30 19:33:17.162736 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:17.162046    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b477c67bfe4ba3eac263d0733c9a53be17d595a21ad9d7ce52b888b282b500cc"
Oct 30 19:33:17.558105 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:17.557786    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f74b9672-t56vz"
Oct 30 19:33:17.627886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:17.621598    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d4da88bf-a2b6-4055-8d91-50f0d10decdd-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d4da88bf-a2b6-4055-8d91-50f0d10decdd-maas-api-openshift-ca\") pod \"d4da88bf-a2b6-4055-8d91-50f0d10decdd\" (UID: \"d4da88bf-a2b6-4055-8d91-50f0d10decdd\") "
Oct 30 19:33:17.627886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:17.621642    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d4da88bf-a2b6-4055-8d91-50f0d10decdd-kube-api-access-jhgz8\" (UniqueName: \"kubernetes.io/projected/d4da88bf-a2b6-4055-8d91-50f0d10decdd-kube-api-access-jhgz8\") pod \"d4da88bf-a2b6-4055-8d91-50f0d10decdd\" (UID: \"d4da88bf-a2b6-4055-8d91-50f0d10decdd\") "
Oct 30 19:33:17.627886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:17.621774    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d4da88bf-a2b6-4055-8d91-50f0d10decdd-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d4da88bf-a2b6-4055-8d91-50f0d10decdd-maas-api-xks-ca\") pod \"d4da88bf-a2b6-4055-8d91-50f0d10decdd\" (UID: \"d4da88bf-a2b6-4055-8d91-50f0d10decdd\") "
Oct 30 19:33:17.674780 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:17.674725    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4da88bf-a2b6-4055-8d91-50f0d10decdd-maas-api-openshift-ca" pod "d4da88bf-a2b6-4055-8d91-50f0d10decdd" (UID: "d4da88bf-a2b6-4055-8d91-50f0d10decdd"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:17.727886 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:33:17.711029    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:33:17.727886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:17.711180    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4da88bf-a2b6-4055-8d91-50f0d10decdd-maas-api-xks-ca" pod "d4da88bf-a2b6-4055-8d91-50f0d10decdd" (UID: "d4da88bf-a2b6-4055-8d91-50f0d10decdd"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:17.727886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:17.721475    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4da88bf-a2b6-4055-8d91-50f0d10decdd-kube-api-access-jhgz8" pod "d4da88bf-a2b6-4055-8d91-50f0d10decdd" (UID: "d4da88bf-a2b6-4055-8d91-50f0d10decdd"). InnerVolumeSpecName "kube-api-access-jhgz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:17.734890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:17.734787    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d4da88bf-a2b6-4055-8d91-50f0d10decdd-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:17.734890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:17.734825    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhgz8\" (UniqueName: \"kubernetes.io/projected/d4da88bf-a2b6-4055-8d91-50f0d10decdd-kube-api-access-jhgz8\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:17.734890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:17.734842    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d4da88bf-a2b6-4055-8d91-50f0d10decdd-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:18.028945 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:18.026301    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 19:33:18.028945 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:18.026363    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc" podUID="e440f6c0-de4c-4a55-af89-7ea0552419a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 19:33:18.173461 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:18.173425    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f74b9672-t56vz"
Oct 30 19:33:18.173461 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:18.173450    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f74b9672-t56vz" event={"ID":"d4da88bf-a2b6-4055-8d91-50f0d10decdd","Type":"ContainerDied","Data":"d69a4c8cca4439640857690699589681cd36e0dd6fe654d3688cbed457b7d466"}
Oct 30 19:33:18.173910 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:18.173531    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d69a4c8cca4439640857690699589681cd36e0dd6fe654d3688cbed457b7d466"
Oct 30 19:33:19.032890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:19.029355    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 19:33:19.032890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:19.029436    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc" podUID="e440f6c0-de4c-4a55-af89-7ea0552419a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 19:33:20.029334 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:20.029289    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 19:33:20.029982 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:20.029947    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc" podUID="e440f6c0-de4c-4a55-af89-7ea0552419a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 19:33:20.445685 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:20.445651    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dbc21c9e-rlz2s"]
Oct 30 19:33:20.446283 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:20.446267    5907 cpu_manager.go:414] "containerMap: removing container" podUID="4c779354-fa02-4c26-999a-bd75c8a3e351" containerName="revoke-keys"
Oct 30 19:33:20.446342 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:20.446285    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4c779354-fa02-4c26-999a-bd75c8a3e351" containerName="revoke-keys"
Oct 30 19:33:20.446342 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:20.446300    5907 cpu_manager.go:414] "containerMap: removing container" podUID="d4da88bf-a2b6-4055-8d91-50f0d10decdd" containerName="revoke-keys"
Oct 30 19:33:20.446342 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:20.446305    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d4da88bf-a2b6-4055-8d91-50f0d10decdd" containerName="revoke-keys"
Oct 30 19:33:20.446436 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:20.446348    5907 cpu_manager.go:414] "containerMap: removing container" podUID="6d1935cb-a979-4aad-aa3f-988c94abd3d6" containerName="revoke-keys"
Oct 30 19:33:20.446436 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:20.446353    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6d1935cb-a979-4aad-aa3f-988c94abd3d6" containerName="revoke-keys"
Oct 30 19:33:20.446436 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:20.446432    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="6d1935cb-a979-4aad-aa3f-988c94abd3d6" containerName="revoke-keys"
Oct 30 19:33:20.446524 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:20.446443    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="4c779354-fa02-4c26-999a-bd75c8a3e351" containerName="revoke-keys"
Oct 30 19:33:20.446524 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:20.446452    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="d4da88bf-a2b6-4055-8d91-50f0d10decdd" containerName="revoke-keys"
Oct 30 19:33:20.449980 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:20.449963    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbc21c9e-rlz2s"
Oct 30 19:33:20.466032 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:20.466011    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:33:20.479575 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:20.479542    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:33:20.482253 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:20.482231    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dbc21c9e-rlz2s"]
Oct 30 19:33:20.573150 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:20.573121    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3638aed3-bb6e-479c-abc8-cc021bd35ca1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dbc21c9e-rlz2s\" (UID: \"3638aed3-bb6e-479c-abc8-cc021bd35ca1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbc21c9e-rlz2s"
Oct 30 19:33:20.573330 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:20.573172    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3638aed3-bb6e-479c-abc8-cc021bd35ca1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dbc21c9e-rlz2s\" (UID: \"3638aed3-bb6e-479c-abc8-cc021bd35ca1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbc21c9e-rlz2s"
Oct 30 19:33:20.573393 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:20.573336    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbjqt\" (UniqueName: \"kubernetes.io/projected/3638aed3-bb6e-479c-abc8-cc021bd35ca1-kube-api-access-mbjqt\") pod \"maas-api-revoke-sub-dbc21c9e-rlz2s\" (UID: \"3638aed3-bb6e-479c-abc8-cc021bd35ca1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbc21c9e-rlz2s"
Oct 30 19:33:20.674096 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:20.674054    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3638aed3-bb6e-479c-abc8-cc021bd35ca1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dbc21c9e-rlz2s\" (UID: \"3638aed3-bb6e-479c-abc8-cc021bd35ca1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbc21c9e-rlz2s"
Oct 30 19:33:20.674297 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:20.674201    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbjqt\" (UniqueName: \"kubernetes.io/projected/3638aed3-bb6e-479c-abc8-cc021bd35ca1-kube-api-access-mbjqt\") pod \"maas-api-revoke-sub-dbc21c9e-rlz2s\" (UID: \"3638aed3-bb6e-479c-abc8-cc021bd35ca1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbc21c9e-rlz2s"
Oct 30 19:33:20.674362 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:20.674327    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3638aed3-bb6e-479c-abc8-cc021bd35ca1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dbc21c9e-rlz2s\" (UID: \"3638aed3-bb6e-479c-abc8-cc021bd35ca1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbc21c9e-rlz2s"
Oct 30 19:33:20.674960 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:20.674936    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3638aed3-bb6e-479c-abc8-cc021bd35ca1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dbc21c9e-rlz2s\" (UID: \"3638aed3-bb6e-479c-abc8-cc021bd35ca1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbc21c9e-rlz2s"
Oct 30 19:33:20.676591 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:20.676571    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3638aed3-bb6e-479c-abc8-cc021bd35ca1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dbc21c9e-rlz2s\" (UID: \"3638aed3-bb6e-479c-abc8-cc021bd35ca1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbc21c9e-rlz2s"
Oct 30 19:33:20.696664 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:20.696610    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbjqt\" (UniqueName: \"kubernetes.io/projected/3638aed3-bb6e-479c-abc8-cc021bd35ca1-kube-api-access-mbjqt\") pod \"maas-api-revoke-sub-dbc21c9e-rlz2s\" (UID: \"3638aed3-bb6e-479c-abc8-cc021bd35ca1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbc21c9e-rlz2s"
Oct 30 19:33:20.763854 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:20.763824    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbc21c9e-rlz2s"
Oct 30 19:33:21.033419 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:21.033327    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 19:33:21.034037 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:21.034004    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc" podUID="e440f6c0-de4c-4a55-af89-7ea0552419a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 19:33:21.093227 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:21.093202    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dbc21c9e-rlz2s"]
Oct 30 19:33:21.094537 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:33:21.094505    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3638aed3_bb6e_479c_abc8_cc021bd35ca1.slice/crio-a13d927d1252543ff376d35db4ed591f304ee507534a30ab371023ef1e827bb8 WatchSource:0}: Error finding container a13d927d1252543ff376d35db4ed591f304ee507534a30ab371023ef1e827bb8: Status 404 returned error can't find the container with id a13d927d1252543ff376d35db4ed591f304ee507534a30ab371023ef1e827bb8
Oct 30 19:33:21.194340 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:21.194299    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbc21c9e-rlz2s" event={"ID":"3638aed3-bb6e-479c-abc8-cc021bd35ca1","Type":"ContainerStarted","Data":"a13d927d1252543ff376d35db4ed591f304ee507534a30ab371023ef1e827bb8"}
Oct 30 19:33:22.202977 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:22.202927    5907 generic.go:356] "Generic (PLEG): container finished" podID="3638aed3-bb6e-479c-abc8-cc021bd35ca1" containerID="859982c47df0da966e7079ed61d40491099540c991c1706c27c2c1d3fa23cdbf" exitCode=0
Oct 30 19:33:22.203386 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:22.203059    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbc21c9e-rlz2s" event={"ID":"3638aed3-bb6e-479c-abc8-cc021bd35ca1","Type":"ContainerDied","Data":"859982c47df0da966e7079ed61d40491099540c991c1706c27c2c1d3fa23cdbf"}
Oct 30 19:33:22.643062 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:22.643023    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-51c63168-vkmwl"]
Oct 30 19:33:22.647112 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:22.647089    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-51c63168-vkmwl"
Oct 30 19:33:22.675813 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:22.675776    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-51c63168-vkmwl"]
Oct 30 19:33:22.712179 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:22.711651    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a071c805-8252-48da-ac16-d4f73a1b15cc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-51c63168-vkmwl\" (UID: \"a071c805-8252-48da-ac16-d4f73a1b15cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-51c63168-vkmwl"
Oct 30 19:33:22.712179 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:22.711892    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccnvg\" (UniqueName: \"kubernetes.io/projected/a071c805-8252-48da-ac16-d4f73a1b15cc-kube-api-access-ccnvg\") pod \"maas-api-revoke-sub-51c63168-vkmwl\" (UID: \"a071c805-8252-48da-ac16-d4f73a1b15cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-51c63168-vkmwl"
Oct 30 19:33:22.712179 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:22.712040    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a071c805-8252-48da-ac16-d4f73a1b15cc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-51c63168-vkmwl\" (UID: \"a071c805-8252-48da-ac16-d4f73a1b15cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-51c63168-vkmwl"
Oct 30 19:33:22.813476 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:22.813438    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a071c805-8252-48da-ac16-d4f73a1b15cc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-51c63168-vkmwl\" (UID: \"a071c805-8252-48da-ac16-d4f73a1b15cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-51c63168-vkmwl"
Oct 30 19:33:22.813772 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:22.813749    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccnvg\" (UniqueName: \"kubernetes.io/projected/a071c805-8252-48da-ac16-d4f73a1b15cc-kube-api-access-ccnvg\") pod \"maas-api-revoke-sub-51c63168-vkmwl\" (UID: \"a071c805-8252-48da-ac16-d4f73a1b15cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-51c63168-vkmwl"
Oct 30 19:33:22.814086 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:22.814066    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a071c805-8252-48da-ac16-d4f73a1b15cc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-51c63168-vkmwl\" (UID: \"a071c805-8252-48da-ac16-d4f73a1b15cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-51c63168-vkmwl"
Oct 30 19:33:22.815199 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:22.815172    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a071c805-8252-48da-ac16-d4f73a1b15cc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-51c63168-vkmwl\" (UID: \"a071c805-8252-48da-ac16-d4f73a1b15cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-51c63168-vkmwl"
Oct 30 19:33:22.816511 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:22.816488    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a071c805-8252-48da-ac16-d4f73a1b15cc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-51c63168-vkmwl\" (UID: \"a071c805-8252-48da-ac16-d4f73a1b15cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-51c63168-vkmwl"
Oct 30 19:33:22.831181 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:22.830981    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"
Oct 30 19:33:22.831739 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:22.831692    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"
Oct 30 19:33:22.833596 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:22.833572    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"
Oct 30 19:33:22.838775 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:22.838756    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccnvg\" (UniqueName: \"kubernetes.io/projected/a071c805-8252-48da-ac16-d4f73a1b15cc-kube-api-access-ccnvg\") pod \"maas-api-revoke-sub-51c63168-vkmwl\" (UID: \"a071c805-8252-48da-ac16-d4f73a1b15cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-51c63168-vkmwl"
Oct 30 19:33:22.962886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:22.962775    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-51c63168-vkmwl"
Oct 30 19:33:23.294633 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:23.294595    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-51c63168-vkmwl"]
Oct 30 19:33:23.295198 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:33:23.294701    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda071c805_8252_48da_ac16_d4f73a1b15cc.slice/crio-4fcf58690e3d8a3922924552fdebd1f31d896616cc6cc62a575c1ebcf6613e11 WatchSource:0}: Error finding container 4fcf58690e3d8a3922924552fdebd1f31d896616cc6cc62a575c1ebcf6613e11: Status 404 returned error can't find the container with id 4fcf58690e3d8a3922924552fdebd1f31d896616cc6cc62a575c1ebcf6613e11
Oct 30 19:33:23.337046 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:23.337018    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbc21c9e-rlz2s"
Oct 30 19:33:23.421944 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:23.421898    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3638aed3-bb6e-479c-abc8-cc021bd35ca1-kube-api-access-mbjqt\" (UniqueName: \"kubernetes.io/projected/3638aed3-bb6e-479c-abc8-cc021bd35ca1-kube-api-access-mbjqt\") pod \"3638aed3-bb6e-479c-abc8-cc021bd35ca1\" (UID: \"3638aed3-bb6e-479c-abc8-cc021bd35ca1\") "
Oct 30 19:33:23.422116 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:23.421994    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3638aed3-bb6e-479c-abc8-cc021bd35ca1-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3638aed3-bb6e-479c-abc8-cc021bd35ca1-maas-api-xks-ca\") pod \"3638aed3-bb6e-479c-abc8-cc021bd35ca1\" (UID: \"3638aed3-bb6e-479c-abc8-cc021bd35ca1\") "
Oct 30 19:33:23.422225 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:23.422206    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3638aed3-bb6e-479c-abc8-cc021bd35ca1-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3638aed3-bb6e-479c-abc8-cc021bd35ca1-maas-api-openshift-ca\") pod \"3638aed3-bb6e-479c-abc8-cc021bd35ca1\" (UID: \"3638aed3-bb6e-479c-abc8-cc021bd35ca1\") "
Oct 30 19:33:23.422814 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:23.422533    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3638aed3-bb6e-479c-abc8-cc021bd35ca1-maas-api-openshift-ca" pod "3638aed3-bb6e-479c-abc8-cc021bd35ca1" (UID: "3638aed3-bb6e-479c-abc8-cc021bd35ca1"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:23.422814 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:23.422780    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3638aed3-bb6e-479c-abc8-cc021bd35ca1-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:23.511028 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:23.510993    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3638aed3-bb6e-479c-abc8-cc021bd35ca1-kube-api-access-mbjqt" pod "3638aed3-bb6e-479c-abc8-cc021bd35ca1" (UID: "3638aed3-bb6e-479c-abc8-cc021bd35ca1"). InnerVolumeSpecName "kube-api-access-mbjqt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:23.511028 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:23.510991    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3638aed3-bb6e-479c-abc8-cc021bd35ca1-maas-api-xks-ca" pod "3638aed3-bb6e-479c-abc8-cc021bd35ca1" (UID: "3638aed3-bb6e-479c-abc8-cc021bd35ca1"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:23.523786 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:23.523712    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbjqt\" (UniqueName: \"kubernetes.io/projected/3638aed3-bb6e-479c-abc8-cc021bd35ca1-kube-api-access-mbjqt\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:23.523786 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:23.523742    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3638aed3-bb6e-479c-abc8-cc021bd35ca1-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:24.221294 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:24.221264    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbc21c9e-rlz2s"
Oct 30 19:33:24.221479 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:24.221302    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbc21c9e-rlz2s" event={"ID":"3638aed3-bb6e-479c-abc8-cc021bd35ca1","Type":"ContainerDied","Data":"a13d927d1252543ff376d35db4ed591f304ee507534a30ab371023ef1e827bb8"}
Oct 30 19:33:24.221479 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:24.221356    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a13d927d1252543ff376d35db4ed591f304ee507534a30ab371023ef1e827bb8"
Oct 30 19:33:24.222905 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:24.222852    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-51c63168-vkmwl" event={"ID":"a071c805-8252-48da-ac16-d4f73a1b15cc","Type":"ContainerStarted","Data":"f3cd177963dc538a0834d5722cd6da98e883f34508829135f595fa34dbe89232"}
Oct 30 19:33:24.223027 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:24.222923    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-51c63168-vkmwl" event={"ID":"a071c805-8252-48da-ac16-d4f73a1b15cc","Type":"ContainerStarted","Data":"4fcf58690e3d8a3922924552fdebd1f31d896616cc6cc62a575c1ebcf6613e11"}
Oct 30 19:33:24.278011 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:24.277959    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-51c63168-vkmwl" podStartSLOduration=2.277943474 podStartE2EDuration="2.277943474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:33:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:33:24.274072212 +0000 UTC m=+716.117004152" watchObservedRunningTime="2026-10-30 19:33:24.277943474 +0000 UTC m=+716.120875412"
Oct 30 19:33:25.228141 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:25.228104    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-379c7cd1-zfmk8"]
Oct 30 19:33:25.228801 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:25.228777    5907 cpu_manager.go:414] "containerMap: removing container" podUID="3638aed3-bb6e-479c-abc8-cc021bd35ca1" containerName="revoke-keys"
Oct 30 19:33:25.228801 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:25.228794    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3638aed3-bb6e-479c-abc8-cc021bd35ca1" containerName="revoke-keys"
Oct 30 19:33:25.228998 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:25.228964    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="3638aed3-bb6e-479c-abc8-cc021bd35ca1" containerName="revoke-keys"
Oct 30 19:33:25.230950 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:25.230924    5907 generic.go:356] "Generic (PLEG): container finished" podID="a071c805-8252-48da-ac16-d4f73a1b15cc" containerID="f3cd177963dc538a0834d5722cd6da98e883f34508829135f595fa34dbe89232" exitCode=0
Oct 30 19:33:25.232546 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:25.232527    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-51c63168-vkmwl" event={"ID":"a071c805-8252-48da-ac16-d4f73a1b15cc","Type":"ContainerDied","Data":"f3cd177963dc538a0834d5722cd6da98e883f34508829135f595fa34dbe89232"}
Oct 30 19:33:25.232617 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:25.232606    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-379c7cd1-zfmk8"
Oct 30 19:33:25.252602 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:25.252576    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-379c7cd1-zfmk8"]
Oct 30 19:33:25.350452 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:25.350416    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qm5px\" (UniqueName: \"kubernetes.io/projected/c5d874c6-d340-4b99-999a-dd4a8922f577-kube-api-access-qm5px\") pod \"maas-api-revoke-sub-379c7cd1-zfmk8\" (UID: \"c5d874c6-d340-4b99-999a-dd4a8922f577\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-379c7cd1-zfmk8"
Oct 30 19:33:25.350630 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:25.350546    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c5d874c6-d340-4b99-999a-dd4a8922f577-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-379c7cd1-zfmk8\" (UID: \"c5d874c6-d340-4b99-999a-dd4a8922f577\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-379c7cd1-zfmk8"
Oct 30 19:33:25.350775 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:25.350761    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c5d874c6-d340-4b99-999a-dd4a8922f577-maas-api-xks-ca\") pod \"maas-api-revoke-sub-379c7cd1-zfmk8\" (UID: \"c5d874c6-d340-4b99-999a-dd4a8922f577\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-379c7cd1-zfmk8"
Oct 30 19:33:25.452007 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:25.451971    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c5d874c6-d340-4b99-999a-dd4a8922f577-maas-api-xks-ca\") pod \"maas-api-revoke-sub-379c7cd1-zfmk8\" (UID: \"c5d874c6-d340-4b99-999a-dd4a8922f577\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-379c7cd1-zfmk8"
Oct 30 19:33:25.452218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:25.452027    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qm5px\" (UniqueName: \"kubernetes.io/projected/c5d874c6-d340-4b99-999a-dd4a8922f577-kube-api-access-qm5px\") pod \"maas-api-revoke-sub-379c7cd1-zfmk8\" (UID: \"c5d874c6-d340-4b99-999a-dd4a8922f577\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-379c7cd1-zfmk8"
Oct 30 19:33:25.452218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:25.452058    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c5d874c6-d340-4b99-999a-dd4a8922f577-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-379c7cd1-zfmk8\" (UID: \"c5d874c6-d340-4b99-999a-dd4a8922f577\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-379c7cd1-zfmk8"
Oct 30 19:33:25.452730 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:25.452706    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c5d874c6-d340-4b99-999a-dd4a8922f577-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-379c7cd1-zfmk8\" (UID: \"c5d874c6-d340-4b99-999a-dd4a8922f577\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-379c7cd1-zfmk8"
Oct 30 19:33:25.454586 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:25.454561    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c5d874c6-d340-4b99-999a-dd4a8922f577-maas-api-xks-ca\") pod \"maas-api-revoke-sub-379c7cd1-zfmk8\" (UID: \"c5d874c6-d340-4b99-999a-dd4a8922f577\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-379c7cd1-zfmk8"
Oct 30 19:33:25.474408 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:25.474376    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qm5px\" (UniqueName: \"kubernetes.io/projected/c5d874c6-d340-4b99-999a-dd4a8922f577-kube-api-access-qm5px\") pod \"maas-api-revoke-sub-379c7cd1-zfmk8\" (UID: \"c5d874c6-d340-4b99-999a-dd4a8922f577\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-379c7cd1-zfmk8"
Oct 30 19:33:25.555824 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:25.555722    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-379c7cd1-zfmk8"
Oct 30 19:33:25.879015 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:25.878986    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-379c7cd1-zfmk8"]
Oct 30 19:33:25.880038 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:33:25.880008    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5d874c6_d340_4b99_999a_dd4a8922f577.slice/crio-926632006e23867187e4592188046380b00d0098b610fb70f0d1a1b928703df4 WatchSource:0}: Error finding container 926632006e23867187e4592188046380b00d0098b610fb70f0d1a1b928703df4: Status 404 returned error can't find the container with id 926632006e23867187e4592188046380b00d0098b610fb70f0d1a1b928703df4
Oct 30 19:33:26.239786 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:26.239748    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-379c7cd1-zfmk8" event={"ID":"c5d874c6-d340-4b99-999a-dd4a8922f577","Type":"ContainerStarted","Data":"cf34a30ed65b866c6697a3b8feb6ccb616de32cf4ca9f455d8e0e24baf762560"}
Oct 30 19:33:26.240184 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:26.239808    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-379c7cd1-zfmk8" event={"ID":"c5d874c6-d340-4b99-999a-dd4a8922f577","Type":"ContainerStarted","Data":"926632006e23867187e4592188046380b00d0098b610fb70f0d1a1b928703df4"}
Oct 30 19:33:26.281141 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:26.281078    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-379c7cd1-zfmk8" podStartSLOduration=1.281060796 podStartE2EDuration="1.281060796s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:33:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:33:26.27999024 +0000 UTC m=+718.122922179" watchObservedRunningTime="2026-10-30 19:33:26.281060796 +0000 UTC m=+718.123992734"
Oct 30 19:33:26.364156 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:26.364128    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-51c63168-vkmwl"
Oct 30 19:33:26.465129 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:26.465104    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a071c805-8252-48da-ac16-d4f73a1b15cc-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a071c805-8252-48da-ac16-d4f73a1b15cc-maas-api-xks-ca\") pod \"a071c805-8252-48da-ac16-d4f73a1b15cc\" (UID: \"a071c805-8252-48da-ac16-d4f73a1b15cc\") "
Oct 30 19:33:26.465279 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:26.465154    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a071c805-8252-48da-ac16-d4f73a1b15cc-kube-api-access-ccnvg\" (UniqueName: \"kubernetes.io/projected/a071c805-8252-48da-ac16-d4f73a1b15cc-kube-api-access-ccnvg\") pod \"a071c805-8252-48da-ac16-d4f73a1b15cc\" (UID: \"a071c805-8252-48da-ac16-d4f73a1b15cc\") "
Oct 30 19:33:26.465279 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:26.465233    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a071c805-8252-48da-ac16-d4f73a1b15cc-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a071c805-8252-48da-ac16-d4f73a1b15cc-maas-api-openshift-ca\") pod \"a071c805-8252-48da-ac16-d4f73a1b15cc\" (UID: \"a071c805-8252-48da-ac16-d4f73a1b15cc\") "
Oct 30 19:33:26.465663 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:26.465635    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a071c805-8252-48da-ac16-d4f73a1b15cc-maas-api-openshift-ca" pod "a071c805-8252-48da-ac16-d4f73a1b15cc" (UID: "a071c805-8252-48da-ac16-d4f73a1b15cc"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:26.467473 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:26.467449    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a071c805-8252-48da-ac16-d4f73a1b15cc-maas-api-xks-ca" pod "a071c805-8252-48da-ac16-d4f73a1b15cc" (UID: "a071c805-8252-48da-ac16-d4f73a1b15cc"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:26.467573 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:26.467495    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a071c805-8252-48da-ac16-d4f73a1b15cc-kube-api-access-ccnvg" pod "a071c805-8252-48da-ac16-d4f73a1b15cc" (UID: "a071c805-8252-48da-ac16-d4f73a1b15cc"). InnerVolumeSpecName "kube-api-access-ccnvg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:26.566253 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:26.566216    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a071c805-8252-48da-ac16-d4f73a1b15cc-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:26.566253 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:26.566248    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ccnvg\" (UniqueName: \"kubernetes.io/projected/a071c805-8252-48da-ac16-d4f73a1b15cc-kube-api-access-ccnvg\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:26.566253 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:26.566258    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a071c805-8252-48da-ac16-d4f73a1b15cc-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:27.018876 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:27.018830    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-fa5a7c92-6vf75"]
Oct 30 19:33:27.019460 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:27.019447    5907 cpu_manager.go:414] "containerMap: removing container" podUID="a071c805-8252-48da-ac16-d4f73a1b15cc" containerName="revoke-keys"
Oct 30 19:33:27.019503 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:27.019462    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a071c805-8252-48da-ac16-d4f73a1b15cc" containerName="revoke-keys"
Oct 30 19:33:27.019567 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:27.019559    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="a071c805-8252-48da-ac16-d4f73a1b15cc" containerName="revoke-keys"
Oct 30 19:33:27.022928 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:27.022911    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fa5a7c92-6vf75"
Oct 30 19:33:27.066201 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:27.066112    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-fa5a7c92-6vf75"]
Oct 30 19:33:27.173262 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:27.173221    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgqqr\" (UniqueName: \"kubernetes.io/projected/ac1c865a-3a93-43d9-a764-902a86c9aa1b-kube-api-access-sgqqr\") pod \"maas-api-revoke-sub-fa5a7c92-6vf75\" (UID: \"ac1c865a-3a93-43d9-a764-902a86c9aa1b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fa5a7c92-6vf75"
Oct 30 19:33:27.173433 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:27.173288    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ac1c865a-3a93-43d9-a764-902a86c9aa1b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-fa5a7c92-6vf75\" (UID: \"ac1c865a-3a93-43d9-a764-902a86c9aa1b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fa5a7c92-6vf75"
Oct 30 19:33:27.173503 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:27.173477    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ac1c865a-3a93-43d9-a764-902a86c9aa1b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-fa5a7c92-6vf75\" (UID: \"ac1c865a-3a93-43d9-a764-902a86c9aa1b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fa5a7c92-6vf75"
Oct 30 19:33:27.247786 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:27.247752    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-51c63168-vkmwl"
Oct 30 19:33:27.248212 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:27.247790    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-51c63168-vkmwl" event={"ID":"a071c805-8252-48da-ac16-d4f73a1b15cc","Type":"ContainerDied","Data":"4fcf58690e3d8a3922924552fdebd1f31d896616cc6cc62a575c1ebcf6613e11"}
Oct 30 19:33:27.248212 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:27.247838    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4fcf58690e3d8a3922924552fdebd1f31d896616cc6cc62a575c1ebcf6613e11"
Oct 30 19:33:27.249619 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:27.249594    5907 generic.go:356] "Generic (PLEG): container finished" podID="c5d874c6-d340-4b99-999a-dd4a8922f577" containerID="cf34a30ed65b866c6697a3b8feb6ccb616de32cf4ca9f455d8e0e24baf762560" exitCode=0
Oct 30 19:33:27.249724 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:27.249622    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-379c7cd1-zfmk8" event={"ID":"c5d874c6-d340-4b99-999a-dd4a8922f577","Type":"ContainerDied","Data":"cf34a30ed65b866c6697a3b8feb6ccb616de32cf4ca9f455d8e0e24baf762560"}
Oct 30 19:33:27.274208 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:27.274179    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ac1c865a-3a93-43d9-a764-902a86c9aa1b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-fa5a7c92-6vf75\" (UID: \"ac1c865a-3a93-43d9-a764-902a86c9aa1b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fa5a7c92-6vf75"
Oct 30 19:33:27.274442 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:27.274293    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgqqr\" (UniqueName: \"kubernetes.io/projected/ac1c865a-3a93-43d9-a764-902a86c9aa1b-kube-api-access-sgqqr\") pod \"maas-api-revoke-sub-fa5a7c92-6vf75\" (UID: \"ac1c865a-3a93-43d9-a764-902a86c9aa1b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fa5a7c92-6vf75"
Oct 30 19:33:27.274442 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:27.274340    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ac1c865a-3a93-43d9-a764-902a86c9aa1b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-fa5a7c92-6vf75\" (UID: \"ac1c865a-3a93-43d9-a764-902a86c9aa1b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fa5a7c92-6vf75"
Oct 30 19:33:27.274919 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:27.274895    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ac1c865a-3a93-43d9-a764-902a86c9aa1b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-fa5a7c92-6vf75\" (UID: \"ac1c865a-3a93-43d9-a764-902a86c9aa1b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fa5a7c92-6vf75"
Oct 30 19:33:27.276829 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:27.276804    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ac1c865a-3a93-43d9-a764-902a86c9aa1b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-fa5a7c92-6vf75\" (UID: \"ac1c865a-3a93-43d9-a764-902a86c9aa1b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fa5a7c92-6vf75"
Oct 30 19:33:27.341937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:27.333041    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgqqr\" (UniqueName: \"kubernetes.io/projected/ac1c865a-3a93-43d9-a764-902a86c9aa1b-kube-api-access-sgqqr\") pod \"maas-api-revoke-sub-fa5a7c92-6vf75\" (UID: \"ac1c865a-3a93-43d9-a764-902a86c9aa1b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fa5a7c92-6vf75"
Oct 30 19:33:27.342558 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:27.342303    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fa5a7c92-6vf75"
Oct 30 19:33:27.649245 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:27.649219    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-fa5a7c92-6vf75"]
Oct 30 19:33:27.651264 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:33:27.651239    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac1c865a_3a93_43d9_a764_902a86c9aa1b.slice/crio-677204efc9a60c622541c5b04c4519e484585e89284d9e2bd1b3646868fa377a WatchSource:0}: Error finding container 677204efc9a60c622541c5b04c4519e484585e89284d9e2bd1b3646868fa377a: Status 404 returned error can't find the container with id 677204efc9a60c622541c5b04c4519e484585e89284d9e2bd1b3646868fa377a
Oct 30 19:33:27.729257 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:27.729229    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-68a9d230-zz95v"]
Oct 30 19:33:27.734584 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:27.734565    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-68a9d230-zz95v"
Oct 30 19:33:27.754372 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:27.754331    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-68a9d230-zz95v"]
Oct 30 19:33:27.882761 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:27.882728    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-68a9d230-zz95v\" (UID: \"cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-68a9d230-zz95v"
Oct 30 19:33:27.882957 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:27.882927    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fgvf\" (UniqueName: \"kubernetes.io/projected/cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a-kube-api-access-2fgvf\") pod \"maas-api-revoke-sub-68a9d230-zz95v\" (UID: \"cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-68a9d230-zz95v"
Oct 30 19:33:27.883015 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:27.882979    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-68a9d230-zz95v\" (UID: \"cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-68a9d230-zz95v"
Oct 30 19:33:27.984468 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:27.984436    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-68a9d230-zz95v\" (UID: \"cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-68a9d230-zz95v"
Oct 30 19:33:27.984638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:27.984541    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fgvf\" (UniqueName: \"kubernetes.io/projected/cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a-kube-api-access-2fgvf\") pod \"maas-api-revoke-sub-68a9d230-zz95v\" (UID: \"cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-68a9d230-zz95v"
Oct 30 19:33:27.984638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:27.984564    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-68a9d230-zz95v\" (UID: \"cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-68a9d230-zz95v"
Oct 30 19:33:27.985231 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:27.985202    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-68a9d230-zz95v\" (UID: \"cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-68a9d230-zz95v"
Oct 30 19:33:27.987244 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:27.987217    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-68a9d230-zz95v\" (UID: \"cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-68a9d230-zz95v"
Oct 30 19:33:28.005025 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:28.004998    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fgvf\" (UniqueName: \"kubernetes.io/projected/cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a-kube-api-access-2fgvf\") pod \"maas-api-revoke-sub-68a9d230-zz95v\" (UID: \"cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-68a9d230-zz95v"
Oct 30 19:33:28.049054 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:28.049016    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-68a9d230-zz95v"
Oct 30 19:33:28.257264 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:28.257152    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fa5a7c92-6vf75" event={"ID":"ac1c865a-3a93-43d9-a764-902a86c9aa1b","Type":"ContainerStarted","Data":"989698a1c05d0495ba62a009c4c96c67b6dcd6de57fb1986c4df1fdf5e3d6131"}
Oct 30 19:33:28.257264 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:28.257234    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fa5a7c92-6vf75" event={"ID":"ac1c865a-3a93-43d9-a764-902a86c9aa1b","Type":"ContainerStarted","Data":"677204efc9a60c622541c5b04c4519e484585e89284d9e2bd1b3646868fa377a"}
Oct 30 19:33:28.299921 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:28.299852    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fa5a7c92-6vf75" podStartSLOduration=2.299839519 podStartE2EDuration="2.299839519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:33:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:33:28.299328806 +0000 UTC m=+720.142260745" watchObservedRunningTime="2026-10-30 19:33:28.299839519 +0000 UTC m=+720.142771457"
Oct 30 19:33:28.367444 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:28.367413    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-68a9d230-zz95v"]
Oct 30 19:33:28.397165 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:28.397142    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-379c7cd1-zfmk8"
Oct 30 19:33:28.490187 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:28.490157    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c5d874c6-d340-4b99-999a-dd4a8922f577-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c5d874c6-d340-4b99-999a-dd4a8922f577-maas-api-openshift-ca\") pod \"c5d874c6-d340-4b99-999a-dd4a8922f577\" (UID: \"c5d874c6-d340-4b99-999a-dd4a8922f577\") "
Oct 30 19:33:28.490375 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:28.490323    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c5d874c6-d340-4b99-999a-dd4a8922f577-kube-api-access-qm5px\" (UniqueName: \"kubernetes.io/projected/c5d874c6-d340-4b99-999a-dd4a8922f577-kube-api-access-qm5px\") pod \"c5d874c6-d340-4b99-999a-dd4a8922f577\" (UID: \"c5d874c6-d340-4b99-999a-dd4a8922f577\") "
Oct 30 19:33:28.490483 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:28.490468    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c5d874c6-d340-4b99-999a-dd4a8922f577-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c5d874c6-d340-4b99-999a-dd4a8922f577-maas-api-xks-ca\") pod \"c5d874c6-d340-4b99-999a-dd4a8922f577\" (UID: \"c5d874c6-d340-4b99-999a-dd4a8922f577\") "
Oct 30 19:33:28.490542 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:28.490506    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5d874c6-d340-4b99-999a-dd4a8922f577-maas-api-openshift-ca" pod "c5d874c6-d340-4b99-999a-dd4a8922f577" (UID: "c5d874c6-d340-4b99-999a-dd4a8922f577"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:28.490845 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:28.490828    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c5d874c6-d340-4b99-999a-dd4a8922f577-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:28.492753 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:28.492711    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5d874c6-d340-4b99-999a-dd4a8922f577-kube-api-access-qm5px" pod "c5d874c6-d340-4b99-999a-dd4a8922f577" (UID: "c5d874c6-d340-4b99-999a-dd4a8922f577"). InnerVolumeSpecName "kube-api-access-qm5px". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:28.492854 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:28.492817    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5d874c6-d340-4b99-999a-dd4a8922f577-maas-api-xks-ca" pod "c5d874c6-d340-4b99-999a-dd4a8922f577" (UID: "c5d874c6-d340-4b99-999a-dd4a8922f577"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:28.541547 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:28.541495    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6323c400-jpdh2"]
Oct 30 19:33:28.542480 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:28.542456    5907 cpu_manager.go:414] "containerMap: removing container" podUID="c5d874c6-d340-4b99-999a-dd4a8922f577" containerName="revoke-keys"
Oct 30 19:33:28.542569 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:28.542484    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c5d874c6-d340-4b99-999a-dd4a8922f577" containerName="revoke-keys"
Oct 30 19:33:28.542677 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:28.542663    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="c5d874c6-d340-4b99-999a-dd4a8922f577" containerName="revoke-keys"
Oct 30 19:33:28.547044 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:28.547024    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6323c400-jpdh2"
Oct 30 19:33:28.567923 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:28.567888    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6323c400-jpdh2"]
Oct 30 19:33:28.592412 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:28.592374    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qm5px\" (UniqueName: \"kubernetes.io/projected/c5d874c6-d340-4b99-999a-dd4a8922f577-kube-api-access-qm5px\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:28.592412 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:28.592406    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c5d874c6-d340-4b99-999a-dd4a8922f577-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:28.693140 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:28.693102    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5b5d8900-05d1-4438-bc0f-2de9c7890ae7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6323c400-jpdh2\" (UID: \"5b5d8900-05d1-4438-bc0f-2de9c7890ae7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6323c400-jpdh2"
Oct 30 19:33:28.693334 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:28.693148    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5b5d8900-05d1-4438-bc0f-2de9c7890ae7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6323c400-jpdh2\" (UID: \"5b5d8900-05d1-4438-bc0f-2de9c7890ae7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6323c400-jpdh2"
Oct 30 19:33:28.693409 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:28.693367    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xm52\" (UniqueName: \"kubernetes.io/projected/5b5d8900-05d1-4438-bc0f-2de9c7890ae7-kube-api-access-8xm52\") pod \"maas-api-revoke-sub-6323c400-jpdh2\" (UID: \"5b5d8900-05d1-4438-bc0f-2de9c7890ae7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6323c400-jpdh2"
Oct 30 19:33:28.794890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:28.794766    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5b5d8900-05d1-4438-bc0f-2de9c7890ae7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6323c400-jpdh2\" (UID: \"5b5d8900-05d1-4438-bc0f-2de9c7890ae7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6323c400-jpdh2"
Oct 30 19:33:28.794890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:28.794807    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5b5d8900-05d1-4438-bc0f-2de9c7890ae7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6323c400-jpdh2\" (UID: \"5b5d8900-05d1-4438-bc0f-2de9c7890ae7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6323c400-jpdh2"
Oct 30 19:33:28.795137 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:28.794941    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xm52\" (UniqueName: \"kubernetes.io/projected/5b5d8900-05d1-4438-bc0f-2de9c7890ae7-kube-api-access-8xm52\") pod \"maas-api-revoke-sub-6323c400-jpdh2\" (UID: \"5b5d8900-05d1-4438-bc0f-2de9c7890ae7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6323c400-jpdh2"
Oct 30 19:33:28.795627 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:28.795604    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5b5d8900-05d1-4438-bc0f-2de9c7890ae7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6323c400-jpdh2\" (UID: \"5b5d8900-05d1-4438-bc0f-2de9c7890ae7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6323c400-jpdh2"
Oct 30 19:33:28.797563 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:28.797541    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5b5d8900-05d1-4438-bc0f-2de9c7890ae7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6323c400-jpdh2\" (UID: \"5b5d8900-05d1-4438-bc0f-2de9c7890ae7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6323c400-jpdh2"
Oct 30 19:33:28.816742 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:28.816708    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xm52\" (UniqueName: \"kubernetes.io/projected/5b5d8900-05d1-4438-bc0f-2de9c7890ae7-kube-api-access-8xm52\") pod \"maas-api-revoke-sub-6323c400-jpdh2\" (UID: \"5b5d8900-05d1-4438-bc0f-2de9c7890ae7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6323c400-jpdh2"
Oct 30 19:33:28.913188 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:28.913141    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6323c400-jpdh2"
Oct 30 19:33:29.242184 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:29.242157    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6323c400-jpdh2"]
Oct 30 19:33:29.243127 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:33:29.243099    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b5d8900_05d1_4438_bc0f_2de9c7890ae7.slice/crio-55b84686c8b986595e6c1c5fa9eda46a6eb27f622da2a67e073419bbd90bf612 WatchSource:0}: Error finding container 55b84686c8b986595e6c1c5fa9eda46a6eb27f622da2a67e073419bbd90bf612: Status 404 returned error can't find the container with id 55b84686c8b986595e6c1c5fa9eda46a6eb27f622da2a67e073419bbd90bf612
Oct 30 19:33:29.264914 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:29.264844    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-68a9d230-zz95v" event={"ID":"cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a","Type":"ContainerStarted","Data":"8da103078d7cd3f790b1c6fc7416de7d6a99fc93b9fb24f09266fb11e896f91e"}
Oct 30 19:33:29.265263 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:29.264944    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-68a9d230-zz95v" event={"ID":"cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a","Type":"ContainerStarted","Data":"67a16c730e9c90885d25763333fe0601e14d8f18a70eec8667e1f8c58ade57eb"}
Oct 30 19:33:29.267039 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:29.267018    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-379c7cd1-zfmk8"
Oct 30 19:33:29.267152 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:29.267059    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-379c7cd1-zfmk8" event={"ID":"c5d874c6-d340-4b99-999a-dd4a8922f577","Type":"ContainerDied","Data":"926632006e23867187e4592188046380b00d0098b610fb70f0d1a1b928703df4"}
Oct 30 19:33:29.267152 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:29.267112    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="926632006e23867187e4592188046380b00d0098b610fb70f0d1a1b928703df4"
Oct 30 19:33:29.268319 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:29.268298    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6323c400-jpdh2" event={"ID":"5b5d8900-05d1-4438-bc0f-2de9c7890ae7","Type":"ContainerStarted","Data":"55b84686c8b986595e6c1c5fa9eda46a6eb27f622da2a67e073419bbd90bf612"}
Oct 30 19:33:29.269888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:29.269842    5907 generic.go:356] "Generic (PLEG): container finished" podID="ac1c865a-3a93-43d9-a764-902a86c9aa1b" containerID="989698a1c05d0495ba62a009c4c96c67b6dcd6de57fb1986c4df1fdf5e3d6131" exitCode=0
Oct 30 19:33:29.269965 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:29.269888    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fa5a7c92-6vf75" event={"ID":"ac1c865a-3a93-43d9-a764-902a86c9aa1b","Type":"ContainerDied","Data":"989698a1c05d0495ba62a009c4c96c67b6dcd6de57fb1986c4df1fdf5e3d6131"}
Oct 30 19:33:29.319948 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:29.319902    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-68a9d230-zz95v" podStartSLOduration=2.319886822 podStartE2EDuration="2.319886822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:33:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:33:29.305253538 +0000 UTC m=+721.148185476" watchObservedRunningTime="2026-10-30 19:33:29.319886822 +0000 UTC m=+721.162818765"
Oct 30 19:33:30.088296 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.088262    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"]
Oct 30 19:33:30.092582 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.092553    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"
Oct 30 19:33:30.108937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.108910    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-f913c6-openshift-default-dockercfg-m5wb4\""
Oct 30 19:33:30.119440 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.119405    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-workload-socket\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"
Oct 30 19:33:30.119615 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.119483    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-workload-certs\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"
Oct 30 19:33:30.119615 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.119593    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-istio-envoy\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"
Oct 30 19:33:30.119764 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.119635    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-istiod-ca-cert\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"
Oct 30 19:33:30.119764 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.119728    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-istio-podinfo\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"
Oct 30 19:33:30.119891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.119795    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-istio-token\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"
Oct 30 19:33:30.119948 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.119928    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-credential-socket\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"
Oct 30 19:33:30.120011 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.119996    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-istio-data\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"
Oct 30 19:33:30.120063 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.120032    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpjw4\" (UniqueName: \"kubernetes.io/projected/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-kube-api-access-cpjw4\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"
Oct 30 19:33:30.126756 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.126720    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"]
Oct 30 19:33:30.221644 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.221619    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-workload-socket\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"
Oct 30 19:33:30.221735 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.221677    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-workload-certs\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"
Oct 30 19:33:30.221735 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.221709    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-istio-envoy\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"
Oct 30 19:33:30.221837 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.221738    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-istiod-ca-cert\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"
Oct 30 19:33:30.221837 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.221767    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-istio-podinfo\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"
Oct 30 19:33:30.221837 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.221817    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-istio-token\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"
Oct 30 19:33:30.222367 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.221903    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-credential-socket\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"
Oct 30 19:33:30.222503 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.222416    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-istio-data\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"
Oct 30 19:33:30.222503 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.222201    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-workload-socket\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"
Oct 30 19:33:30.222503 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.222455    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpjw4\" (UniqueName: \"kubernetes.io/projected/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-kube-api-access-cpjw4\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"
Oct 30 19:33:30.222503 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.222223    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-workload-certs\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"
Oct 30 19:33:30.222503 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.222340    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-credential-socket\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"
Oct 30 19:33:30.222966 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.222692    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-istio-data\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"
Oct 30 19:33:30.222966 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.222717    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-istiod-ca-cert\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"
Oct 30 19:33:30.224494 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.224472    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-istio-envoy\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"
Oct 30 19:33:30.224782 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.224760    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-istio-podinfo\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"
Oct 30 19:33:30.243169 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.243145    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-istio-token\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"
Oct 30 19:33:30.243169 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.243177    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpjw4\" (UniqueName: \"kubernetes.io/projected/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-kube-api-access-cpjw4\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"
Oct 30 19:33:30.278994 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.278958    5907 generic.go:356] "Generic (PLEG): container finished" podID="cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a" containerID="8da103078d7cd3f790b1c6fc7416de7d6a99fc93b9fb24f09266fb11e896f91e" exitCode=0
Oct 30 19:33:30.279407 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.279027    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-68a9d230-zz95v" event={"ID":"cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a","Type":"ContainerDied","Data":"8da103078d7cd3f790b1c6fc7416de7d6a99fc93b9fb24f09266fb11e896f91e"}
Oct 30 19:33:30.281065 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.281027    5907 generic.go:356] "Generic (PLEG): container finished" podID="5b5d8900-05d1-4438-bc0f-2de9c7890ae7" containerID="432a6fddc5664657187e7164d6a05254d4fc9a35e6ec9928d80a510894254087" exitCode=0
Oct 30 19:33:30.291999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.291963    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6323c400-jpdh2" event={"ID":"5b5d8900-05d1-4438-bc0f-2de9c7890ae7","Type":"ContainerDied","Data":"432a6fddc5664657187e7164d6a05254d4fc9a35e6ec9928d80a510894254087"}
Oct 30 19:33:30.412311 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.412130    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"
Oct 30 19:33:30.483900 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.483850    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fa5a7c92-6vf75"
Oct 30 19:33:30.533094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.533061    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ac1c865a-3a93-43d9-a764-902a86c9aa1b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ac1c865a-3a93-43d9-a764-902a86c9aa1b-maas-api-openshift-ca\") pod \"ac1c865a-3a93-43d9-a764-902a86c9aa1b\" (UID: \"ac1c865a-3a93-43d9-a764-902a86c9aa1b\") "
Oct 30 19:33:30.533291 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.533158    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ac1c865a-3a93-43d9-a764-902a86c9aa1b-kube-api-access-sgqqr\" (UniqueName: \"kubernetes.io/projected/ac1c865a-3a93-43d9-a764-902a86c9aa1b-kube-api-access-sgqqr\") pod \"ac1c865a-3a93-43d9-a764-902a86c9aa1b\" (UID: \"ac1c865a-3a93-43d9-a764-902a86c9aa1b\") "
Oct 30 19:33:30.533291 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.533243    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ac1c865a-3a93-43d9-a764-902a86c9aa1b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ac1c865a-3a93-43d9-a764-902a86c9aa1b-maas-api-xks-ca\") pod \"ac1c865a-3a93-43d9-a764-902a86c9aa1b\" (UID: \"ac1c865a-3a93-43d9-a764-902a86c9aa1b\") "
Oct 30 19:33:30.534797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.534743    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac1c865a-3a93-43d9-a764-902a86c9aa1b-maas-api-openshift-ca" pod "ac1c865a-3a93-43d9-a764-902a86c9aa1b" (UID: "ac1c865a-3a93-43d9-a764-902a86c9aa1b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:30.546183 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.546149    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac1c865a-3a93-43d9-a764-902a86c9aa1b-maas-api-xks-ca" pod "ac1c865a-3a93-43d9-a764-902a86c9aa1b" (UID: "ac1c865a-3a93-43d9-a764-902a86c9aa1b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:30.546345 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.546165    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac1c865a-3a93-43d9-a764-902a86c9aa1b-kube-api-access-sgqqr" pod "ac1c865a-3a93-43d9-a764-902a86c9aa1b" (UID: "ac1c865a-3a93-43d9-a764-902a86c9aa1b"). InnerVolumeSpecName "kube-api-access-sgqqr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:30.634745 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.634712    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sgqqr\" (UniqueName: \"kubernetes.io/projected/ac1c865a-3a93-43d9-a764-902a86c9aa1b-kube-api-access-sgqqr\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:30.634745 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.634742    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ac1c865a-3a93-43d9-a764-902a86c9aa1b-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:30.634745 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.634751    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ac1c865a-3a93-43d9-a764-902a86c9aa1b-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:30.762453 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.762422    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"]
Oct 30 19:33:30.763657 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:33:30.763622    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fbf8d00_57a5_49ba_855a_4accc2f1bb05.slice/crio-a0f42be8d64860d5ed99b63af2bf44d6b93a01f52580facba2036ddcf3ec569d WatchSource:0}: Error finding container a0f42be8d64860d5ed99b63af2bf44d6b93a01f52580facba2036ddcf3ec569d: Status 404 returned error can't find the container with id a0f42be8d64860d5ed99b63af2bf44d6b93a01f52580facba2036ddcf3ec569d
Oct 30 19:33:30.766143 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.766109    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:33:30.766239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.766176    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:33:30.766239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:30.766206    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:33:31.295044 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:31.295013    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fa5a7c92-6vf75"
Oct 30 19:33:31.295044 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:31.295029    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fa5a7c92-6vf75" event={"ID":"ac1c865a-3a93-43d9-a764-902a86c9aa1b","Type":"ContainerDied","Data":"677204efc9a60c622541c5b04c4519e484585e89284d9e2bd1b3646868fa377a"}
Oct 30 19:33:31.295600 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:31.295086    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="677204efc9a60c622541c5b04c4519e484585e89284d9e2bd1b3646868fa377a"
Oct 30 19:33:31.296767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:31.296739    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5" event={"ID":"0fbf8d00-57a5-49ba-855a-4accc2f1bb05","Type":"ContainerStarted","Data":"fbb146431899285ec8a1f0d33e1164a44016cd50160aa548a1d66672cd7b50b7"}
Oct 30 19:33:31.296942 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:31.296788    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5" event={"ID":"0fbf8d00-57a5-49ba-855a-4accc2f1bb05","Type":"ContainerStarted","Data":"a0f42be8d64860d5ed99b63af2bf44d6b93a01f52580facba2036ddcf3ec569d"}
Oct 30 19:33:31.366525 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:31.366467    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5" podStartSLOduration=1.366449147 podStartE2EDuration="1.366449147s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:33:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:33:31.359187148 +0000 UTC m=+723.202119088" watchObservedRunningTime="2026-10-30 19:33:31.366449147 +0000 UTC m=+723.209381085"
Oct 30 19:33:31.483287 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:31.480015    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"
Oct 30 19:33:31.512930 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:31.504195    5907 patch_prober.go:29] interesting pod/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 30 19:33:31.512930 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:31.504252    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5" podUID="0fbf8d00-57a5-49ba-855a-4accc2f1bb05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 30 19:33:32.068886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.059442    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-68a9d230-zz95v"
Oct 30 19:33:32.109936 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.109900    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a-kube-api-access-2fgvf\" (UniqueName: \"kubernetes.io/projected/cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a-kube-api-access-2fgvf\") pod \"cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a\" (UID: \"cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a\") "
Oct 30 19:33:32.114909 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.110385    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6323c400-jpdh2"
Oct 30 19:33:32.120464 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.120433    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a-maas-api-xks-ca\") pod \"cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a\" (UID: \"cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a\") "
Oct 30 19:33:32.126124 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.126043    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a-maas-api-openshift-ca" pod "cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a" (UID: "cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:32.229915 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.219192    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a-kube-api-access-2fgvf" pod "cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a" (UID: "cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a"). InnerVolumeSpecName "kube-api-access-2fgvf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:32.229915 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.221002    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a-maas-api-openshift-ca\") pod \"cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a\" (UID: \"cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a\") "
Oct 30 19:33:32.325956 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.325838    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5b5d8900-05d1-4438-bc0f-2de9c7890ae7-kube-api-access-8xm52\" (UniqueName: \"kubernetes.io/projected/5b5d8900-05d1-4438-bc0f-2de9c7890ae7-kube-api-access-8xm52\") pod \"5b5d8900-05d1-4438-bc0f-2de9c7890ae7\" (UID: \"5b5d8900-05d1-4438-bc0f-2de9c7890ae7\") "
Oct 30 19:33:32.343007 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.342179    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5b5d8900-05d1-4438-bc0f-2de9c7890ae7-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5b5d8900-05d1-4438-bc0f-2de9c7890ae7-maas-api-xks-ca\") pod \"5b5d8900-05d1-4438-bc0f-2de9c7890ae7\" (UID: \"5b5d8900-05d1-4438-bc0f-2de9c7890ae7\") "
Oct 30 19:33:32.343007 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.342252    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5b5d8900-05d1-4438-bc0f-2de9c7890ae7-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5b5d8900-05d1-4438-bc0f-2de9c7890ae7-maas-api-openshift-ca\") pod \"5b5d8900-05d1-4438-bc0f-2de9c7890ae7\" (UID: \"5b5d8900-05d1-4438-bc0f-2de9c7890ae7\") "
Oct 30 19:33:32.343572 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.343497    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2fgvf\" (UniqueName: \"kubernetes.io/projected/cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a-kube-api-access-2fgvf\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:32.343572 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.343532    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:32.351993 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.351917    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b5d8900-05d1-4438-bc0f-2de9c7890ae7-maas-api-openshift-ca" pod "5b5d8900-05d1-4438-bc0f-2de9c7890ae7" (UID: "5b5d8900-05d1-4438-bc0f-2de9c7890ae7"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:32.410134 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.410028    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b5d8900-05d1-4438-bc0f-2de9c7890ae7-kube-api-access-8xm52" pod "5b5d8900-05d1-4438-bc0f-2de9c7890ae7" (UID: "5b5d8900-05d1-4438-bc0f-2de9c7890ae7"). InnerVolumeSpecName "kube-api-access-8xm52". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:32.420928 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.414661    5907 patch_prober.go:29] interesting pod/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 30 19:33:32.420928 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.414721    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5" podUID="0fbf8d00-57a5-49ba-855a-4accc2f1bb05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 30 19:33:32.435099 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.435066    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a-maas-api-xks-ca" pod "cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a" (UID: "cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:32.451671 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.450900    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xm52\" (UniqueName: \"kubernetes.io/projected/5b5d8900-05d1-4438-bc0f-2de9c7890ae7-kube-api-access-8xm52\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:32.451671 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.450955    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:32.451671 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.450974    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5b5d8900-05d1-4438-bc0f-2de9c7890ae7-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:32.463057 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.461138    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b5d8900-05d1-4438-bc0f-2de9c7890ae7-maas-api-xks-ca" pod "5b5d8900-05d1-4438-bc0f-2de9c7890ae7" (UID: "5b5d8900-05d1-4438-bc0f-2de9c7890ae7"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:32.506421 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.500512    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-68a9d230-zz95v" event={"ID":"cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a","Type":"ContainerDied","Data":"67a16c730e9c90885d25763333fe0601e14d8f18a70eec8667e1f8c58ade57eb"}
Oct 30 19:33:32.506421 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.500579    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="67a16c730e9c90885d25763333fe0601e14d8f18a70eec8667e1f8c58ade57eb"
Oct 30 19:33:32.506421 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.500663    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-68a9d230-zz95v"
Oct 30 19:33:32.539667 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.539636    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6323c400-jpdh2"
Oct 30 19:33:32.549646 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.546708    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6323c400-jpdh2" event={"ID":"5b5d8900-05d1-4438-bc0f-2de9c7890ae7","Type":"ContainerDied","Data":"55b84686c8b986595e6c1c5fa9eda46a6eb27f622da2a67e073419bbd90bf612"}
Oct 30 19:33:32.550315 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.549842    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="55b84686c8b986595e6c1c5fa9eda46a6eb27f622da2a67e073419bbd90bf612"
Oct 30 19:33:32.578713 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.578680    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5b5d8900-05d1-4438-bc0f-2de9c7890ae7-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:32.962889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.962370    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1913ec02-q8sd5"]
Oct 30 19:33:32.966893 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.963290    5907 cpu_manager.go:414] "containerMap: removing container" podUID="5b5d8900-05d1-4438-bc0f-2de9c7890ae7" containerName="revoke-keys"
Oct 30 19:33:32.966893 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.963313    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5b5d8900-05d1-4438-bc0f-2de9c7890ae7" containerName="revoke-keys"
Oct 30 19:33:32.966893 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.963344    5907 cpu_manager.go:414] "containerMap: removing container" podUID="ac1c865a-3a93-43d9-a764-902a86c9aa1b" containerName="revoke-keys"
Oct 30 19:33:32.966893 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.963352    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ac1c865a-3a93-43d9-a764-902a86c9aa1b" containerName="revoke-keys"
Oct 30 19:33:32.966893 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.963370    5907 cpu_manager.go:414] "containerMap: removing container" podUID="cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a" containerName="revoke-keys"
Oct 30 19:33:32.966893 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.963377    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a" containerName="revoke-keys"
Oct 30 19:33:32.966893 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.963511    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="5b5d8900-05d1-4438-bc0f-2de9c7890ae7" containerName="revoke-keys"
Oct 30 19:33:32.966893 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.963528    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="ac1c865a-3a93-43d9-a764-902a86c9aa1b" containerName="revoke-keys"
Oct 30 19:33:32.966893 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.963539    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a" containerName="revoke-keys"
Oct 30 19:33:32.982888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:32.982258    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1913ec02-q8sd5"
Oct 30 19:33:33.007139 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:33.007110    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:33:33.007517 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:33.007496    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:33:33.080188 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:33.080152    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1913ec02-q8sd5"]
Oct 30 19:33:33.140889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:33.140787    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e346d798-b892-42b0-9832-ffb14d72f22a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1913ec02-q8sd5\" (UID: \"e346d798-b892-42b0-9832-ffb14d72f22a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1913ec02-q8sd5"
Oct 30 19:33:33.140889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:33.140846    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg6qw\" (UniqueName: \"kubernetes.io/projected/e346d798-b892-42b0-9832-ffb14d72f22a-kube-api-access-kg6qw\") pod \"maas-api-revoke-sub-1913ec02-q8sd5\" (UID: \"e346d798-b892-42b0-9832-ffb14d72f22a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1913ec02-q8sd5"
Oct 30 19:33:33.141137 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:33.140928    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e346d798-b892-42b0-9832-ffb14d72f22a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1913ec02-q8sd5\" (UID: \"e346d798-b892-42b0-9832-ffb14d72f22a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1913ec02-q8sd5"
Oct 30 19:33:33.198887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:33.198299    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-010785bc-9prhn"]
Oct 30 19:33:33.280242 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:33.280202    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e346d798-b892-42b0-9832-ffb14d72f22a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1913ec02-q8sd5\" (UID: \"e346d798-b892-42b0-9832-ffb14d72f22a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1913ec02-q8sd5"
Oct 30 19:33:33.280490 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:33.280465    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg6qw\" (UniqueName: \"kubernetes.io/projected/e346d798-b892-42b0-9832-ffb14d72f22a-kube-api-access-kg6qw\") pod \"maas-api-revoke-sub-1913ec02-q8sd5\" (UID: \"e346d798-b892-42b0-9832-ffb14d72f22a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1913ec02-q8sd5"
Oct 30 19:33:33.280704 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:33.280684    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e346d798-b892-42b0-9832-ffb14d72f22a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1913ec02-q8sd5\" (UID: \"e346d798-b892-42b0-9832-ffb14d72f22a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1913ec02-q8sd5"
Oct 30 19:33:33.281904 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:33.281879    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e346d798-b892-42b0-9832-ffb14d72f22a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1913ec02-q8sd5\" (UID: \"e346d798-b892-42b0-9832-ffb14d72f22a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1913ec02-q8sd5"
Oct 30 19:33:33.286947 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:33.282105    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-010785bc-9prhn"
Oct 30 19:33:33.288303 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:33.283835    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-010785bc-9prhn"]
Oct 30 19:33:33.294406 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:33.294040    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e346d798-b892-42b0-9832-ffb14d72f22a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1913ec02-q8sd5\" (UID: \"e346d798-b892-42b0-9832-ffb14d72f22a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1913ec02-q8sd5"
Oct 30 19:33:33.322885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:33.319676    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg6qw\" (UniqueName: \"kubernetes.io/projected/e346d798-b892-42b0-9832-ffb14d72f22a-kube-api-access-kg6qw\") pod \"maas-api-revoke-sub-1913ec02-q8sd5\" (UID: \"e346d798-b892-42b0-9832-ffb14d72f22a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1913ec02-q8sd5"
Oct 30 19:33:33.382453 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:33.382419    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jb66w\" (UniqueName: \"kubernetes.io/projected/588869d6-5b48-4d49-8c18-4d91a187c7db-kube-api-access-jb66w\") pod \"maas-api-revoke-sub-010785bc-9prhn\" (UID: \"588869d6-5b48-4d49-8c18-4d91a187c7db\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-010785bc-9prhn"
Oct 30 19:33:33.382983 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:33.382961    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/588869d6-5b48-4d49-8c18-4d91a187c7db-maas-api-xks-ca\") pod \"maas-api-revoke-sub-010785bc-9prhn\" (UID: \"588869d6-5b48-4d49-8c18-4d91a187c7db\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-010785bc-9prhn"
Oct 30 19:33:33.383215 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:33.383199    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/588869d6-5b48-4d49-8c18-4d91a187c7db-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-010785bc-9prhn\" (UID: \"588869d6-5b48-4d49-8c18-4d91a187c7db\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-010785bc-9prhn"
Oct 30 19:33:33.414945 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:33.414393    5907 patch_prober.go:29] interesting pod/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 30 19:33:33.414945 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:33.414471    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5" podUID="0fbf8d00-57a5-49ba-855a-4accc2f1bb05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 30 19:33:33.485832 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:33.484921    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/588869d6-5b48-4d49-8c18-4d91a187c7db-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-010785bc-9prhn\" (UID: \"588869d6-5b48-4d49-8c18-4d91a187c7db\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-010785bc-9prhn"
Oct 30 19:33:33.486129 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:33.486107    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jb66w\" (UniqueName: \"kubernetes.io/projected/588869d6-5b48-4d49-8c18-4d91a187c7db-kube-api-access-jb66w\") pod \"maas-api-revoke-sub-010785bc-9prhn\" (UID: \"588869d6-5b48-4d49-8c18-4d91a187c7db\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-010785bc-9prhn"
Oct 30 19:33:33.486717 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:33.486694    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/588869d6-5b48-4d49-8c18-4d91a187c7db-maas-api-xks-ca\") pod \"maas-api-revoke-sub-010785bc-9prhn\" (UID: \"588869d6-5b48-4d49-8c18-4d91a187c7db\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-010785bc-9prhn"
Oct 30 19:33:33.487407 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:33.487081    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/588869d6-5b48-4d49-8c18-4d91a187c7db-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-010785bc-9prhn\" (UID: \"588869d6-5b48-4d49-8c18-4d91a187c7db\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-010785bc-9prhn"
Oct 30 19:33:33.490045 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:33.490024    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/588869d6-5b48-4d49-8c18-4d91a187c7db-maas-api-xks-ca\") pod \"maas-api-revoke-sub-010785bc-9prhn\" (UID: \"588869d6-5b48-4d49-8c18-4d91a187c7db\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-010785bc-9prhn"
Oct 30 19:33:33.514886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:33.514018    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jb66w\" (UniqueName: \"kubernetes.io/projected/588869d6-5b48-4d49-8c18-4d91a187c7db-kube-api-access-jb66w\") pod \"maas-api-revoke-sub-010785bc-9prhn\" (UID: \"588869d6-5b48-4d49-8c18-4d91a187c7db\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-010785bc-9prhn"
Oct 30 19:33:33.572071 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:33.572025    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1913ec02-q8sd5"
Oct 30 19:33:33.681448 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:33.681067    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-010785bc-9prhn"
Oct 30 19:33:34.450888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:34.443206    5907 patch_prober.go:29] interesting pod/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 30 19:33:34.450888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:34.443265    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5" podUID="0fbf8d00-57a5-49ba-855a-4accc2f1bb05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 30 19:33:34.688978 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:34.663305    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1913ec02-q8sd5"]
Oct 30 19:33:34.880718 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:34.880682    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1913ec02-q8sd5" event={"ID":"e346d798-b892-42b0-9832-ffb14d72f22a","Type":"ContainerStarted","Data":"9d6d78f865d184062bc37d0312995bcab87759cc62b59c4da7de47bdc4f3fb4b"}
Oct 30 19:33:35.284367 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:35.278213    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-010785bc-9prhn"]
Oct 30 19:33:35.428321 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:35.427682    5907 patch_prober.go:29] interesting pod/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 30 19:33:35.428321 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:35.427737    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5" podUID="0fbf8d00-57a5-49ba-855a-4accc2f1bb05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 30 19:33:35.889090 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:35.889055    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-010785bc-9prhn" event={"ID":"588869d6-5b48-4d49-8c18-4d91a187c7db","Type":"ContainerStarted","Data":"2f98c98e6b9104629c935ea6b0d73421bdb4a8b4e5518fddefe9d6de9d0a4a0e"}
Oct 30 19:33:35.889494 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:35.889108    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-010785bc-9prhn" event={"ID":"588869d6-5b48-4d49-8c18-4d91a187c7db","Type":"ContainerStarted","Data":"169a4bd53b1d9a5b76181b9bcf6afc46c3bf21104c314925359a7a118c688fb2"}
Oct 30 19:33:35.890715 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:35.890691    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1913ec02-q8sd5" event={"ID":"e346d798-b892-42b0-9832-ffb14d72f22a","Type":"ContainerStarted","Data":"48d8d2e419c68453738064eefb299e31572ff50c4d074e309e60bfd2d43129ff"}
Oct 30 19:33:35.978524 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:35.978393    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1913ec02-q8sd5" podStartSLOduration=3.978376043 podStartE2EDuration="3.978376043s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:33:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:33:35.978153989 +0000 UTC m=+727.821085943" watchObservedRunningTime="2026-10-30 19:33:35.978376043 +0000 UTC m=+727.821307983"
Oct 30 19:33:35.978802 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:35.978772    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-010785bc-9prhn" podStartSLOduration=2.9787616359999998 podStartE2EDuration="2.978761636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:33:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:33:35.938179139 +0000 UTC m=+727.781111079" watchObservedRunningTime="2026-10-30 19:33:35.978761636 +0000 UTC m=+727.821693575"
Oct 30 19:33:36.427580 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:36.422957    5907 patch_prober.go:29] interesting pod/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 30 19:33:36.427580 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:36.423016    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5" podUID="0fbf8d00-57a5-49ba-855a-4accc2f1bb05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 30 19:33:36.899486 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:36.899450    5907 generic.go:356] "Generic (PLEG): container finished" podID="e346d798-b892-42b0-9832-ffb14d72f22a" containerID="48d8d2e419c68453738064eefb299e31572ff50c4d074e309e60bfd2d43129ff" exitCode=0
Oct 30 19:33:36.899945 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:36.899540    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1913ec02-q8sd5" event={"ID":"e346d798-b892-42b0-9832-ffb14d72f22a","Type":"ContainerDied","Data":"48d8d2e419c68453738064eefb299e31572ff50c4d074e309e60bfd2d43129ff"}
Oct 30 19:33:36.901473 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:36.901449    5907 generic.go:356] "Generic (PLEG): container finished" podID="588869d6-5b48-4d49-8c18-4d91a187c7db" containerID="2f98c98e6b9104629c935ea6b0d73421bdb4a8b4e5518fddefe9d6de9d0a4a0e" exitCode=0
Oct 30 19:33:36.901617 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:36.901534    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-010785bc-9prhn" event={"ID":"588869d6-5b48-4d49-8c18-4d91a187c7db","Type":"ContainerDied","Data":"2f98c98e6b9104629c935ea6b0d73421bdb4a8b4e5518fddefe9d6de9d0a4a0e"}
Oct 30 19:33:37.418322 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:37.418279    5907 patch_prober.go:29] interesting pod/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 30 19:33:37.418610 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:37.418580    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5" podUID="0fbf8d00-57a5-49ba-855a-4accc2f1bb05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 30 19:33:37.660843 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:37.660804    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a9c1b2dc-whzhk"]
Oct 30 19:33:37.665533 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:37.665513    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a9c1b2dc-whzhk"
Oct 30 19:33:37.691909 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:37.691849    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a9c1b2dc-whzhk"]
Oct 30 19:33:37.762220 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:37.762179    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z78zv\" (UniqueName: \"kubernetes.io/projected/af57f262-1206-4414-bbb7-7aa4aca88391-kube-api-access-z78zv\") pod \"maas-api-revoke-sub-a9c1b2dc-whzhk\" (UID: \"af57f262-1206-4414-bbb7-7aa4aca88391\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a9c1b2dc-whzhk"
Oct 30 19:33:37.762391 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:37.762234    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/af57f262-1206-4414-bbb7-7aa4aca88391-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a9c1b2dc-whzhk\" (UID: \"af57f262-1206-4414-bbb7-7aa4aca88391\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a9c1b2dc-whzhk"
Oct 30 19:33:37.762436 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:37.762386    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/af57f262-1206-4414-bbb7-7aa4aca88391-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a9c1b2dc-whzhk\" (UID: \"af57f262-1206-4414-bbb7-7aa4aca88391\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a9c1b2dc-whzhk"
Oct 30 19:33:37.864203 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:37.864169    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/af57f262-1206-4414-bbb7-7aa4aca88391-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a9c1b2dc-whzhk\" (UID: \"af57f262-1206-4414-bbb7-7aa4aca88391\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a9c1b2dc-whzhk"
Oct 30 19:33:37.864370 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:37.864302    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z78zv\" (UniqueName: \"kubernetes.io/projected/af57f262-1206-4414-bbb7-7aa4aca88391-kube-api-access-z78zv\") pod \"maas-api-revoke-sub-a9c1b2dc-whzhk\" (UID: \"af57f262-1206-4414-bbb7-7aa4aca88391\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a9c1b2dc-whzhk"
Oct 30 19:33:37.864370 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:37.864336    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/af57f262-1206-4414-bbb7-7aa4aca88391-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a9c1b2dc-whzhk\" (UID: \"af57f262-1206-4414-bbb7-7aa4aca88391\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a9c1b2dc-whzhk"
Oct 30 19:33:37.864786 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:37.864765    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/af57f262-1206-4414-bbb7-7aa4aca88391-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a9c1b2dc-whzhk\" (UID: \"af57f262-1206-4414-bbb7-7aa4aca88391\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a9c1b2dc-whzhk"
Oct 30 19:33:37.866838 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:37.866814    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/af57f262-1206-4414-bbb7-7aa4aca88391-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a9c1b2dc-whzhk\" (UID: \"af57f262-1206-4414-bbb7-7aa4aca88391\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a9c1b2dc-whzhk"
Oct 30 19:33:37.884986 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:37.884965    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z78zv\" (UniqueName: \"kubernetes.io/projected/af57f262-1206-4414-bbb7-7aa4aca88391-kube-api-access-z78zv\") pod \"maas-api-revoke-sub-a9c1b2dc-whzhk\" (UID: \"af57f262-1206-4414-bbb7-7aa4aca88391\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a9c1b2dc-whzhk"
Oct 30 19:33:37.988605 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:37.985541    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a9c1b2dc-whzhk"
Oct 30 19:33:38.086959 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:38.086591    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-010785bc-9prhn"
Oct 30 19:33:38.113140 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:38.113042    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1913ec02-q8sd5"
Oct 30 19:33:38.272039 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:38.271921    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e346d798-b892-42b0-9832-ffb14d72f22a-kube-api-access-kg6qw\" (UniqueName: \"kubernetes.io/projected/e346d798-b892-42b0-9832-ffb14d72f22a-kube-api-access-kg6qw\") pod \"e346d798-b892-42b0-9832-ffb14d72f22a\" (UID: \"e346d798-b892-42b0-9832-ffb14d72f22a\") "
Oct 30 19:33:38.272039 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:38.272011    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/588869d6-5b48-4d49-8c18-4d91a187c7db-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/588869d6-5b48-4d49-8c18-4d91a187c7db-maas-api-openshift-ca\") pod \"588869d6-5b48-4d49-8c18-4d91a187c7db\" (UID: \"588869d6-5b48-4d49-8c18-4d91a187c7db\") "
Oct 30 19:33:38.272302 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:38.272039    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/588869d6-5b48-4d49-8c18-4d91a187c7db-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/588869d6-5b48-4d49-8c18-4d91a187c7db-maas-api-xks-ca\") pod \"588869d6-5b48-4d49-8c18-4d91a187c7db\" (UID: \"588869d6-5b48-4d49-8c18-4d91a187c7db\") "
Oct 30 19:33:38.272302 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:38.272110    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e346d798-b892-42b0-9832-ffb14d72f22a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e346d798-b892-42b0-9832-ffb14d72f22a-maas-api-xks-ca\") pod \"e346d798-b892-42b0-9832-ffb14d72f22a\" (UID: \"e346d798-b892-42b0-9832-ffb14d72f22a\") "
Oct 30 19:33:38.272302 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:38.272146    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/588869d6-5b48-4d49-8c18-4d91a187c7db-kube-api-access-jb66w\" (UniqueName: \"kubernetes.io/projected/588869d6-5b48-4d49-8c18-4d91a187c7db-kube-api-access-jb66w\") pod \"588869d6-5b48-4d49-8c18-4d91a187c7db\" (UID: \"588869d6-5b48-4d49-8c18-4d91a187c7db\") "
Oct 30 19:33:38.272302 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:38.272182    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e346d798-b892-42b0-9832-ffb14d72f22a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e346d798-b892-42b0-9832-ffb14d72f22a-maas-api-openshift-ca\") pod \"e346d798-b892-42b0-9832-ffb14d72f22a\" (UID: \"e346d798-b892-42b0-9832-ffb14d72f22a\") "
Oct 30 19:33:38.272520 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:38.272434    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/588869d6-5b48-4d49-8c18-4d91a187c7db-maas-api-openshift-ca" pod "588869d6-5b48-4d49-8c18-4d91a187c7db" (UID: "588869d6-5b48-4d49-8c18-4d91a187c7db"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:38.272948 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:38.272921    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/588869d6-5b48-4d49-8c18-4d91a187c7db-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:38.273207 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:38.273184    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e346d798-b892-42b0-9832-ffb14d72f22a-maas-api-openshift-ca" pod "e346d798-b892-42b0-9832-ffb14d72f22a" (UID: "e346d798-b892-42b0-9832-ffb14d72f22a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:38.274843 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:38.274816    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e346d798-b892-42b0-9832-ffb14d72f22a-kube-api-access-kg6qw" pod "e346d798-b892-42b0-9832-ffb14d72f22a" (UID: "e346d798-b892-42b0-9832-ffb14d72f22a"). InnerVolumeSpecName "kube-api-access-kg6qw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:38.275057 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:38.274956    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/588869d6-5b48-4d49-8c18-4d91a187c7db-maas-api-xks-ca" pod "588869d6-5b48-4d49-8c18-4d91a187c7db" (UID: "588869d6-5b48-4d49-8c18-4d91a187c7db"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:38.275332 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:38.275312    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/588869d6-5b48-4d49-8c18-4d91a187c7db-kube-api-access-jb66w" pod "588869d6-5b48-4d49-8c18-4d91a187c7db" (UID: "588869d6-5b48-4d49-8c18-4d91a187c7db"). InnerVolumeSpecName "kube-api-access-jb66w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:38.275756 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:38.275739    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e346d798-b892-42b0-9832-ffb14d72f22a-maas-api-xks-ca" pod "e346d798-b892-42b0-9832-ffb14d72f22a" (UID: "e346d798-b892-42b0-9832-ffb14d72f22a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:38.370842 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:38.370813    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a9c1b2dc-whzhk"]
Oct 30 19:33:38.371447 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:33:38.371422    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf57f262_1206_4414_bbb7_7aa4aca88391.slice/crio-85cad48a6fbe0282288c1e9a4aec0ca7ded98722e7c8c4a63fa57c7435900c00 WatchSource:0}: Error finding container 85cad48a6fbe0282288c1e9a4aec0ca7ded98722e7c8c4a63fa57c7435900c00: Status 404 returned error can't find the container with id 85cad48a6fbe0282288c1e9a4aec0ca7ded98722e7c8c4a63fa57c7435900c00
Oct 30 19:33:38.373766 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:38.373741    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/588869d6-5b48-4d49-8c18-4d91a187c7db-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:38.373766 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:38.373765    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e346d798-b892-42b0-9832-ffb14d72f22a-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:38.373902 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:38.373774    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jb66w\" (UniqueName: \"kubernetes.io/projected/588869d6-5b48-4d49-8c18-4d91a187c7db-kube-api-access-jb66w\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:38.373902 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:38.373783    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e346d798-b892-42b0-9832-ffb14d72f22a-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:38.373902 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:38.373793    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kg6qw\" (UniqueName: \"kubernetes.io/projected/e346d798-b892-42b0-9832-ffb14d72f22a-kube-api-access-kg6qw\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:38.919122 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:38.919085    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a9c1b2dc-whzhk" event={"ID":"af57f262-1206-4414-bbb7-7aa4aca88391","Type":"ContainerStarted","Data":"3ef07aa89761b08ac6bd7a20aed449b36a88ba5dcd57da4254749d55a3343afb"}
Oct 30 19:33:38.919301 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:38.919155    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a9c1b2dc-whzhk" event={"ID":"af57f262-1206-4414-bbb7-7aa4aca88391","Type":"ContainerStarted","Data":"85cad48a6fbe0282288c1e9a4aec0ca7ded98722e7c8c4a63fa57c7435900c00"}
Oct 30 19:33:38.920843 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:38.920809    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-010785bc-9prhn" event={"ID":"588869d6-5b48-4d49-8c18-4d91a187c7db","Type":"ContainerDied","Data":"169a4bd53b1d9a5b76181b9bcf6afc46c3bf21104c314925359a7a118c688fb2"}
Oct 30 19:33:38.921029 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:38.920880    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="169a4bd53b1d9a5b76181b9bcf6afc46c3bf21104c314925359a7a118c688fb2"
Oct 30 19:33:38.921029 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:38.920825    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-010785bc-9prhn"
Oct 30 19:33:38.922502 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:38.922476    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1913ec02-q8sd5" event={"ID":"e346d798-b892-42b0-9832-ffb14d72f22a","Type":"ContainerDied","Data":"9d6d78f865d184062bc37d0312995bcab87759cc62b59c4da7de47bdc4f3fb4b"}
Oct 30 19:33:38.922597 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:38.922508    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1913ec02-q8sd5"
Oct 30 19:33:38.922597 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:38.922524    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9d6d78f865d184062bc37d0312995bcab87759cc62b59c4da7de47bdc4f3fb4b"
Oct 30 19:33:38.962217 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:38.962159    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a9c1b2dc-whzhk" podStartSLOduration=1.962144715 podStartE2EDuration="1.962144715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:33:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:33:38.95972495 +0000 UTC m=+730.802656883" watchObservedRunningTime="2026-10-30 19:33:38.962144715 +0000 UTC m=+730.805076689"
Oct 30 19:33:39.110318 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:39.110287    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"
Oct 30 19:33:39.110803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:39.110674    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"
Oct 30 19:33:39.111604 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:39.111582    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"
Oct 30 19:33:39.930381 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:39.930349    5907 generic.go:356] "Generic (PLEG): container finished" podID="af57f262-1206-4414-bbb7-7aa4aca88391" containerID="3ef07aa89761b08ac6bd7a20aed449b36a88ba5dcd57da4254749d55a3343afb" exitCode=0
Oct 30 19:33:39.930586 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:39.930443    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a9c1b2dc-whzhk" event={"ID":"af57f262-1206-4414-bbb7-7aa4aca88391","Type":"ContainerDied","Data":"3ef07aa89761b08ac6bd7a20aed449b36a88ba5dcd57da4254749d55a3343afb"}
Oct 30 19:33:40.992845 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:40.992810    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0c4a8c8d-v6k4n"]
Oct 30 19:33:40.993567 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:40.993548    5907 cpu_manager.go:414] "containerMap: removing container" podUID="e346d798-b892-42b0-9832-ffb14d72f22a" containerName="revoke-keys"
Oct 30 19:33:40.993620 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:40.993569    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e346d798-b892-42b0-9832-ffb14d72f22a" containerName="revoke-keys"
Oct 30 19:33:40.993620 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:40.993589    5907 cpu_manager.go:414] "containerMap: removing container" podUID="588869d6-5b48-4d49-8c18-4d91a187c7db" containerName="revoke-keys"
Oct 30 19:33:40.993620 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:40.993596    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="588869d6-5b48-4d49-8c18-4d91a187c7db" containerName="revoke-keys"
Oct 30 19:33:40.993753 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:40.993740    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="e346d798-b892-42b0-9832-ffb14d72f22a" containerName="revoke-keys"
Oct 30 19:33:40.993792 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:40.993755    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="588869d6-5b48-4d49-8c18-4d91a187c7db" containerName="revoke-keys"
Oct 30 19:33:40.998219 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:40.998183    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c4a8c8d-v6k4n"
Oct 30 19:33:41.006619 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.006589    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0be07a33-84b6-4d99-ae4e-ef83769bd359-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0c4a8c8d-v6k4n\" (UID: \"0be07a33-84b6-4d99-ae4e-ef83769bd359\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c4a8c8d-v6k4n"
Oct 30 19:33:41.006992 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.006969    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr52x\" (UniqueName: \"kubernetes.io/projected/0be07a33-84b6-4d99-ae4e-ef83769bd359-kube-api-access-hr52x\") pod \"maas-api-revoke-sub-0c4a8c8d-v6k4n\" (UID: \"0be07a33-84b6-4d99-ae4e-ef83769bd359\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c4a8c8d-v6k4n"
Oct 30 19:33:41.007182 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.007163    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0be07a33-84b6-4d99-ae4e-ef83769bd359-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0c4a8c8d-v6k4n\" (UID: \"0be07a33-84b6-4d99-ae4e-ef83769bd359\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c4a8c8d-v6k4n"
Oct 30 19:33:41.026077 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.026039    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ef1c16f2-4c9d5"]
Oct 30 19:33:41.030925 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.030900    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ef1c16f2-4c9d5"
Oct 30 19:33:41.036214 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.036187    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0c4a8c8d-v6k4n"]
Oct 30 19:33:41.076486 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.076451    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ef1c16f2-4c9d5"]
Oct 30 19:33:41.106980 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.106950    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a9c1b2dc-whzhk"
Oct 30 19:33:41.108194 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.108167    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0be07a33-84b6-4d99-ae4e-ef83769bd359-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0c4a8c8d-v6k4n\" (UID: \"0be07a33-84b6-4d99-ae4e-ef83769bd359\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c4a8c8d-v6k4n"
Oct 30 19:33:41.108327 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.108228    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0be07a33-84b6-4d99-ae4e-ef83769bd359-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0c4a8c8d-v6k4n\" (UID: \"0be07a33-84b6-4d99-ae4e-ef83769bd359\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c4a8c8d-v6k4n"
Oct 30 19:33:41.108327 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.108318    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fd223aca-6c79-4fb0-8946-c37bddd8f6be-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ef1c16f2-4c9d5\" (UID: \"fd223aca-6c79-4fb0-8946-c37bddd8f6be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ef1c16f2-4c9d5"
Oct 30 19:33:41.108433 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.108366    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s48z7\" (UniqueName: \"kubernetes.io/projected/fd223aca-6c79-4fb0-8946-c37bddd8f6be-kube-api-access-s48z7\") pod \"maas-api-revoke-sub-ef1c16f2-4c9d5\" (UID: \"fd223aca-6c79-4fb0-8946-c37bddd8f6be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ef1c16f2-4c9d5"
Oct 30 19:33:41.108433 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.108426    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fd223aca-6c79-4fb0-8946-c37bddd8f6be-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ef1c16f2-4c9d5\" (UID: \"fd223aca-6c79-4fb0-8946-c37bddd8f6be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ef1c16f2-4c9d5"
Oct 30 19:33:41.108563 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.108547    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr52x\" (UniqueName: \"kubernetes.io/projected/0be07a33-84b6-4d99-ae4e-ef83769bd359-kube-api-access-hr52x\") pod \"maas-api-revoke-sub-0c4a8c8d-v6k4n\" (UID: \"0be07a33-84b6-4d99-ae4e-ef83769bd359\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c4a8c8d-v6k4n"
Oct 30 19:33:41.109513 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.109477    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0be07a33-84b6-4d99-ae4e-ef83769bd359-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0c4a8c8d-v6k4n\" (UID: \"0be07a33-84b6-4d99-ae4e-ef83769bd359\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c4a8c8d-v6k4n"
Oct 30 19:33:41.111051 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.111029    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0be07a33-84b6-4d99-ae4e-ef83769bd359-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0c4a8c8d-v6k4n\" (UID: \"0be07a33-84b6-4d99-ae4e-ef83769bd359\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c4a8c8d-v6k4n"
Oct 30 19:33:41.128760 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.128730    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr52x\" (UniqueName: \"kubernetes.io/projected/0be07a33-84b6-4d99-ae4e-ef83769bd359-kube-api-access-hr52x\") pod \"maas-api-revoke-sub-0c4a8c8d-v6k4n\" (UID: \"0be07a33-84b6-4d99-ae4e-ef83769bd359\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c4a8c8d-v6k4n"
Oct 30 19:33:41.209183 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.209102    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af57f262-1206-4414-bbb7-7aa4aca88391-kube-api-access-z78zv\" (UniqueName: \"kubernetes.io/projected/af57f262-1206-4414-bbb7-7aa4aca88391-kube-api-access-z78zv\") pod \"af57f262-1206-4414-bbb7-7aa4aca88391\" (UID: \"af57f262-1206-4414-bbb7-7aa4aca88391\") "
Oct 30 19:33:41.209183 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.209150    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/af57f262-1206-4414-bbb7-7aa4aca88391-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/af57f262-1206-4414-bbb7-7aa4aca88391-maas-api-openshift-ca\") pod \"af57f262-1206-4414-bbb7-7aa4aca88391\" (UID: \"af57f262-1206-4414-bbb7-7aa4aca88391\") "
Oct 30 19:33:41.209423 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.209186    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/af57f262-1206-4414-bbb7-7aa4aca88391-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/af57f262-1206-4414-bbb7-7aa4aca88391-maas-api-xks-ca\") pod \"af57f262-1206-4414-bbb7-7aa4aca88391\" (UID: \"af57f262-1206-4414-bbb7-7aa4aca88391\") "
Oct 30 19:33:41.209423 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.209333    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fd223aca-6c79-4fb0-8946-c37bddd8f6be-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ef1c16f2-4c9d5\" (UID: \"fd223aca-6c79-4fb0-8946-c37bddd8f6be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ef1c16f2-4c9d5"
Oct 30 19:33:41.209423 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.209382    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s48z7\" (UniqueName: \"kubernetes.io/projected/fd223aca-6c79-4fb0-8946-c37bddd8f6be-kube-api-access-s48z7\") pod \"maas-api-revoke-sub-ef1c16f2-4c9d5\" (UID: \"fd223aca-6c79-4fb0-8946-c37bddd8f6be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ef1c16f2-4c9d5"
Oct 30 19:33:41.209603 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.209439    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fd223aca-6c79-4fb0-8946-c37bddd8f6be-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ef1c16f2-4c9d5\" (UID: \"fd223aca-6c79-4fb0-8946-c37bddd8f6be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ef1c16f2-4c9d5"
Oct 30 19:33:41.209654 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.209609    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af57f262-1206-4414-bbb7-7aa4aca88391-maas-api-openshift-ca" pod "af57f262-1206-4414-bbb7-7aa4aca88391" (UID: "af57f262-1206-4414-bbb7-7aa4aca88391"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:41.209838 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.209755    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/af57f262-1206-4414-bbb7-7aa4aca88391-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:41.210109 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.210089    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fd223aca-6c79-4fb0-8946-c37bddd8f6be-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ef1c16f2-4c9d5\" (UID: \"fd223aca-6c79-4fb0-8946-c37bddd8f6be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ef1c16f2-4c9d5"
Oct 30 19:33:41.211500 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.211475    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af57f262-1206-4414-bbb7-7aa4aca88391-kube-api-access-z78zv" pod "af57f262-1206-4414-bbb7-7aa4aca88391" (UID: "af57f262-1206-4414-bbb7-7aa4aca88391"). InnerVolumeSpecName "kube-api-access-z78zv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:41.211627 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.211609    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af57f262-1206-4414-bbb7-7aa4aca88391-maas-api-xks-ca" pod "af57f262-1206-4414-bbb7-7aa4aca88391" (UID: "af57f262-1206-4414-bbb7-7aa4aca88391"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:41.212036 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.212020    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fd223aca-6c79-4fb0-8946-c37bddd8f6be-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ef1c16f2-4c9d5\" (UID: \"fd223aca-6c79-4fb0-8946-c37bddd8f6be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ef1c16f2-4c9d5"
Oct 30 19:33:41.230519 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.230494    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s48z7\" (UniqueName: \"kubernetes.io/projected/fd223aca-6c79-4fb0-8946-c37bddd8f6be-kube-api-access-s48z7\") pod \"maas-api-revoke-sub-ef1c16f2-4c9d5\" (UID: \"fd223aca-6c79-4fb0-8946-c37bddd8f6be\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ef1c16f2-4c9d5"
Oct 30 19:33:41.310666 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.310638    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z78zv\" (UniqueName: \"kubernetes.io/projected/af57f262-1206-4414-bbb7-7aa4aca88391-kube-api-access-z78zv\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:41.310666 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.310663    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/af57f262-1206-4414-bbb7-7aa4aca88391-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:41.316115 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.316096    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c4a8c8d-v6k4n"
Oct 30 19:33:41.352358 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.352328    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ef1c16f2-4c9d5"
Oct 30 19:33:41.637085 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.637058    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0c4a8c8d-v6k4n"]
Oct 30 19:33:41.638114 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:33:41.638077    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0be07a33_84b6_4d99_ae4e_ef83769bd359.slice/crio-f4e713b2f92e9a9e61487cf576657c3ba90a5662aa942a417e9fa842ad951939 WatchSource:0}: Error finding container f4e713b2f92e9a9e61487cf576657c3ba90a5662aa942a417e9fa842ad951939: Status 404 returned error can't find the container with id f4e713b2f92e9a9e61487cf576657c3ba90a5662aa942a417e9fa842ad951939
Oct 30 19:33:41.677950 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.677919    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ef1c16f2-4c9d5"]
Oct 30 19:33:41.678143 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:33:41.678120    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd223aca_6c79_4fb0_8946_c37bddd8f6be.slice/crio-ef884cdcd204f3cd80a77b2d4541f69e143b92ed3b380388692e431647c785a9 WatchSource:0}: Error finding container ef884cdcd204f3cd80a77b2d4541f69e143b92ed3b380388692e431647c785a9: Status 404 returned error can't find the container with id ef884cdcd204f3cd80a77b2d4541f69e143b92ed3b380388692e431647c785a9
Oct 30 19:33:41.944965 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.944841    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ef1c16f2-4c9d5" event={"ID":"fd223aca-6c79-4fb0-8946-c37bddd8f6be","Type":"ContainerStarted","Data":"ef884cdcd204f3cd80a77b2d4541f69e143b92ed3b380388692e431647c785a9"}
Oct 30 19:33:41.946158 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.946129    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c4a8c8d-v6k4n" event={"ID":"0be07a33-84b6-4d99-ae4e-ef83769bd359","Type":"ContainerStarted","Data":"f4e713b2f92e9a9e61487cf576657c3ba90a5662aa942a417e9fa842ad951939"}
Oct 30 19:33:41.947349 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.947329    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a9c1b2dc-whzhk" event={"ID":"af57f262-1206-4414-bbb7-7aa4aca88391","Type":"ContainerDied","Data":"85cad48a6fbe0282288c1e9a4aec0ca7ded98722e7c8c4a63fa57c7435900c00"}
Oct 30 19:33:41.947449 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.947363    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="85cad48a6fbe0282288c1e9a4aec0ca7ded98722e7c8c4a63fa57c7435900c00"
Oct 30 19:33:41.947449 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:41.947335    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a9c1b2dc-whzhk"
Oct 30 19:33:42.956630 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:42.956591    5907 generic.go:356] "Generic (PLEG): container finished" podID="fd223aca-6c79-4fb0-8946-c37bddd8f6be" containerID="1b1c0063983ad87aa52f461c6c974456982e7f4911322ba92b2b7b91d2ac16be" exitCode=0
Oct 30 19:33:42.957077 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:42.956677    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ef1c16f2-4c9d5" event={"ID":"fd223aca-6c79-4fb0-8946-c37bddd8f6be","Type":"ContainerDied","Data":"1b1c0063983ad87aa52f461c6c974456982e7f4911322ba92b2b7b91d2ac16be"}
Oct 30 19:33:42.958390 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:42.958369    5907 generic.go:356] "Generic (PLEG): container finished" podID="0be07a33-84b6-4d99-ae4e-ef83769bd359" containerID="1939eda93c46b905411846a2bb139993377326ffb3d5187095d33be9ff178a40" exitCode=0
Oct 30 19:33:42.958528 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:42.958402    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c4a8c8d-v6k4n" event={"ID":"0be07a33-84b6-4d99-ae4e-ef83769bd359","Type":"ContainerDied","Data":"1939eda93c46b905411846a2bb139993377326ffb3d5187095d33be9ff178a40"}
Oct 30 19:33:44.132041 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:44.131974    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c4a8c8d-v6k4n"
Oct 30 19:33:44.149974 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:44.149946    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0be07a33-84b6-4d99-ae4e-ef83769bd359-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0be07a33-84b6-4d99-ae4e-ef83769bd359-maas-api-openshift-ca\") pod \"0be07a33-84b6-4d99-ae4e-ef83769bd359\" (UID: \"0be07a33-84b6-4d99-ae4e-ef83769bd359\") "
Oct 30 19:33:44.150096 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:44.149993    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0be07a33-84b6-4d99-ae4e-ef83769bd359-kube-api-access-hr52x\" (UniqueName: \"kubernetes.io/projected/0be07a33-84b6-4d99-ae4e-ef83769bd359-kube-api-access-hr52x\") pod \"0be07a33-84b6-4d99-ae4e-ef83769bd359\" (UID: \"0be07a33-84b6-4d99-ae4e-ef83769bd359\") "
Oct 30 19:33:44.150096 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:44.150084    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0be07a33-84b6-4d99-ae4e-ef83769bd359-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0be07a33-84b6-4d99-ae4e-ef83769bd359-maas-api-xks-ca\") pod \"0be07a33-84b6-4d99-ae4e-ef83769bd359\" (UID: \"0be07a33-84b6-4d99-ae4e-ef83769bd359\") "
Oct 30 19:33:44.150324 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:44.150301    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0be07a33-84b6-4d99-ae4e-ef83769bd359-maas-api-openshift-ca" pod "0be07a33-84b6-4d99-ae4e-ef83769bd359" (UID: "0be07a33-84b6-4d99-ae4e-ef83769bd359"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:44.150451 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:44.150434    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0be07a33-84b6-4d99-ae4e-ef83769bd359-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:44.152340 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:44.152316    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0be07a33-84b6-4d99-ae4e-ef83769bd359-kube-api-access-hr52x" pod "0be07a33-84b6-4d99-ae4e-ef83769bd359" (UID: "0be07a33-84b6-4d99-ae4e-ef83769bd359"). InnerVolumeSpecName "kube-api-access-hr52x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:44.152419 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:44.152337    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0be07a33-84b6-4d99-ae4e-ef83769bd359-maas-api-xks-ca" pod "0be07a33-84b6-4d99-ae4e-ef83769bd359" (UID: "0be07a33-84b6-4d99-ae4e-ef83769bd359"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:44.169109 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:44.169091    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ef1c16f2-4c9d5"
Oct 30 19:33:44.260144 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:44.260050    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fd223aca-6c79-4fb0-8946-c37bddd8f6be-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fd223aca-6c79-4fb0-8946-c37bddd8f6be-maas-api-xks-ca\") pod \"fd223aca-6c79-4fb0-8946-c37bddd8f6be\" (UID: \"fd223aca-6c79-4fb0-8946-c37bddd8f6be\") "
Oct 30 19:33:44.260144 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:44.260136    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fd223aca-6c79-4fb0-8946-c37bddd8f6be-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fd223aca-6c79-4fb0-8946-c37bddd8f6be-maas-api-openshift-ca\") pod \"fd223aca-6c79-4fb0-8946-c37bddd8f6be\" (UID: \"fd223aca-6c79-4fb0-8946-c37bddd8f6be\") "
Oct 30 19:33:44.260379 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:44.260361    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd223aca-6c79-4fb0-8946-c37bddd8f6be-kube-api-access-s48z7\" (UniqueName: \"kubernetes.io/projected/fd223aca-6c79-4fb0-8946-c37bddd8f6be-kube-api-access-s48z7\") pod \"fd223aca-6c79-4fb0-8946-c37bddd8f6be\" (UID: \"fd223aca-6c79-4fb0-8946-c37bddd8f6be\") "
Oct 30 19:33:44.261180 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:44.261157    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hr52x\" (UniqueName: \"kubernetes.io/projected/0be07a33-84b6-4d99-ae4e-ef83769bd359-kube-api-access-hr52x\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:44.261265 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:44.261187    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0be07a33-84b6-4d99-ae4e-ef83769bd359-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:44.262227 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:44.262191    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd223aca-6c79-4fb0-8946-c37bddd8f6be-maas-api-openshift-ca" pod "fd223aca-6c79-4fb0-8946-c37bddd8f6be" (UID: "fd223aca-6c79-4fb0-8946-c37bddd8f6be"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:44.264076 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:44.264049    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd223aca-6c79-4fb0-8946-c37bddd8f6be-kube-api-access-s48z7" pod "fd223aca-6c79-4fb0-8946-c37bddd8f6be" (UID: "fd223aca-6c79-4fb0-8946-c37bddd8f6be"). InnerVolumeSpecName "kube-api-access-s48z7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:44.264201 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:44.264172    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd223aca-6c79-4fb0-8946-c37bddd8f6be-maas-api-xks-ca" pod "fd223aca-6c79-4fb0-8946-c37bddd8f6be" (UID: "fd223aca-6c79-4fb0-8946-c37bddd8f6be"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:44.362169 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:44.362133    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s48z7\" (UniqueName: \"kubernetes.io/projected/fd223aca-6c79-4fb0-8946-c37bddd8f6be-kube-api-access-s48z7\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:44.362169 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:44.362164    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fd223aca-6c79-4fb0-8946-c37bddd8f6be-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:44.362169 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:44.362174    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fd223aca-6c79-4fb0-8946-c37bddd8f6be-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:44.973252 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:44.973219    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ef1c16f2-4c9d5"
Oct 30 19:33:44.973444 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:44.973257    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ef1c16f2-4c9d5" event={"ID":"fd223aca-6c79-4fb0-8946-c37bddd8f6be","Type":"ContainerDied","Data":"ef884cdcd204f3cd80a77b2d4541f69e143b92ed3b380388692e431647c785a9"}
Oct 30 19:33:44.973444 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:44.973312    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ef884cdcd204f3cd80a77b2d4541f69e143b92ed3b380388692e431647c785a9"
Oct 30 19:33:44.974719 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:44.974696    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c4a8c8d-v6k4n"
Oct 30 19:33:44.974719 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:44.974712    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c4a8c8d-v6k4n" event={"ID":"0be07a33-84b6-4d99-ae4e-ef83769bd359","Type":"ContainerDied","Data":"f4e713b2f92e9a9e61487cf576657c3ba90a5662aa942a417e9fa842ad951939"}
Oct 30 19:33:44.974914 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:44.974750    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f4e713b2f92e9a9e61487cf576657c3ba90a5662aa942a417e9fa842ad951939"
Oct 30 19:33:45.850961 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:45.850927    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:33:46.138215 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:46.138182    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3eb9ce78-524x8"]
Oct 30 19:33:46.138749 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:46.138737    5907 cpu_manager.go:414] "containerMap: removing container" podUID="0be07a33-84b6-4d99-ae4e-ef83769bd359" containerName="revoke-keys"
Oct 30 19:33:46.138797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:46.138750    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0be07a33-84b6-4d99-ae4e-ef83769bd359" containerName="revoke-keys"
Oct 30 19:33:46.138797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:46.138773    5907 cpu_manager.go:414] "containerMap: removing container" podUID="af57f262-1206-4414-bbb7-7aa4aca88391" containerName="revoke-keys"
Oct 30 19:33:46.138797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:46.138779    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="af57f262-1206-4414-bbb7-7aa4aca88391" containerName="revoke-keys"
Oct 30 19:33:46.138797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:46.138792    5907 cpu_manager.go:414] "containerMap: removing container" podUID="fd223aca-6c79-4fb0-8946-c37bddd8f6be" containerName="revoke-keys"
Oct 30 19:33:46.138797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:46.138797    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fd223aca-6c79-4fb0-8946-c37bddd8f6be" containerName="revoke-keys"
Oct 30 19:33:46.138960 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:46.138896    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="0be07a33-84b6-4d99-ae4e-ef83769bd359" containerName="revoke-keys"
Oct 30 19:33:46.138960 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:46.138908    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="af57f262-1206-4414-bbb7-7aa4aca88391" containerName="revoke-keys"
Oct 30 19:33:46.138960 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:46.138916    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="fd223aca-6c79-4fb0-8946-c37bddd8f6be" containerName="revoke-keys"
Oct 30 19:33:46.142506 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:46.142490    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3eb9ce78-524x8"
Oct 30 19:33:46.163708 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:46.163672    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3eb9ce78-524x8"]
Oct 30 19:33:46.172498 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:46.172459    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:33:46.172639 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:46.172540    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:33:46.182113 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:46.182088    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ad250179-35bc-4289-9732-91a9ab45ab67-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3eb9ce78-524x8\" (UID: \"ad250179-35bc-4289-9732-91a9ab45ab67\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3eb9ce78-524x8"
Oct 30 19:33:46.182359 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:46.182336    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm9pl\" (UniqueName: \"kubernetes.io/projected/ad250179-35bc-4289-9732-91a9ab45ab67-kube-api-access-hm9pl\") pod \"maas-api-revoke-sub-3eb9ce78-524x8\" (UID: \"ad250179-35bc-4289-9732-91a9ab45ab67\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3eb9ce78-524x8"
Oct 30 19:33:46.182470 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:46.182450    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ad250179-35bc-4289-9732-91a9ab45ab67-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3eb9ce78-524x8\" (UID: \"ad250179-35bc-4289-9732-91a9ab45ab67\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3eb9ce78-524x8"
Oct 30 19:33:46.283161 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:46.283130    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ad250179-35bc-4289-9732-91a9ab45ab67-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3eb9ce78-524x8\" (UID: \"ad250179-35bc-4289-9732-91a9ab45ab67\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3eb9ce78-524x8"
Oct 30 19:33:46.283336 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:46.283272    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm9pl\" (UniqueName: \"kubernetes.io/projected/ad250179-35bc-4289-9732-91a9ab45ab67-kube-api-access-hm9pl\") pod \"maas-api-revoke-sub-3eb9ce78-524x8\" (UID: \"ad250179-35bc-4289-9732-91a9ab45ab67\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3eb9ce78-524x8"
Oct 30 19:33:46.283336 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:46.283319    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ad250179-35bc-4289-9732-91a9ab45ab67-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3eb9ce78-524x8\" (UID: \"ad250179-35bc-4289-9732-91a9ab45ab67\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3eb9ce78-524x8"
Oct 30 19:33:46.283988 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:46.283961    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ad250179-35bc-4289-9732-91a9ab45ab67-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3eb9ce78-524x8\" (UID: \"ad250179-35bc-4289-9732-91a9ab45ab67\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3eb9ce78-524x8"
Oct 30 19:33:46.285892 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:46.285853    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ad250179-35bc-4289-9732-91a9ab45ab67-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3eb9ce78-524x8\" (UID: \"ad250179-35bc-4289-9732-91a9ab45ab67\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3eb9ce78-524x8"
Oct 30 19:33:46.304699 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:46.304668    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm9pl\" (UniqueName: \"kubernetes.io/projected/ad250179-35bc-4289-9732-91a9ab45ab67-kube-api-access-hm9pl\") pod \"maas-api-revoke-sub-3eb9ce78-524x8\" (UID: \"ad250179-35bc-4289-9732-91a9ab45ab67\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3eb9ce78-524x8"
Oct 30 19:33:46.460272 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:46.460179    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3eb9ce78-524x8"
Oct 30 19:33:46.857218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:46.857187    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3eb9ce78-524x8"]
Oct 30 19:33:46.858166 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:33:46.858135    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad250179_35bc_4289_9732_91a9ab45ab67.slice/crio-f28303a90cea3ed5b46ce21c2007a87e2c288994d566aa6df432df1f0b444775 WatchSource:0}: Error finding container f28303a90cea3ed5b46ce21c2007a87e2c288994d566aa6df432df1f0b444775: Status 404 returned error can't find the container with id f28303a90cea3ed5b46ce21c2007a87e2c288994d566aa6df432df1f0b444775
Oct 30 19:33:47.010259 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:47.010220    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3eb9ce78-524x8" event={"ID":"ad250179-35bc-4289-9732-91a9ab45ab67","Type":"ContainerStarted","Data":"f28303a90cea3ed5b46ce21c2007a87e2c288994d566aa6df432df1f0b444775"}
Oct 30 19:33:47.828889 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:33:47.826372    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:33:48.054727 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:48.054684    5907 generic.go:356] "Generic (PLEG): container finished" podID="ad250179-35bc-4289-9732-91a9ab45ab67" containerID="8f310e7abe98f21ae923d552c8a1efa4fb92a2582f65ce8be692726341f50579" exitCode=0
Oct 30 19:33:48.055375 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:48.055351    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3eb9ce78-524x8" event={"ID":"ad250179-35bc-4289-9732-91a9ab45ab67","Type":"ContainerDied","Data":"8f310e7abe98f21ae923d552c8a1efa4fb92a2582f65ce8be692726341f50579"}
Oct 30 19:33:49.269839 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:49.269813    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3eb9ce78-524x8"
Oct 30 19:33:49.276578 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:49.275882    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ad250179-35bc-4289-9732-91a9ab45ab67-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ad250179-35bc-4289-9732-91a9ab45ab67-maas-api-xks-ca\") pod \"ad250179-35bc-4289-9732-91a9ab45ab67\" (UID: \"ad250179-35bc-4289-9732-91a9ab45ab67\") "
Oct 30 19:33:49.276578 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:49.275978    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ad250179-35bc-4289-9732-91a9ab45ab67-kube-api-access-hm9pl\" (UniqueName: \"kubernetes.io/projected/ad250179-35bc-4289-9732-91a9ab45ab67-kube-api-access-hm9pl\") pod \"ad250179-35bc-4289-9732-91a9ab45ab67\" (UID: \"ad250179-35bc-4289-9732-91a9ab45ab67\") "
Oct 30 19:33:49.282398 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:49.282352    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad250179-35bc-4289-9732-91a9ab45ab67-kube-api-access-hm9pl" pod "ad250179-35bc-4289-9732-91a9ab45ab67" (UID: "ad250179-35bc-4289-9732-91a9ab45ab67"). InnerVolumeSpecName "kube-api-access-hm9pl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:49.282529 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:49.282431    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad250179-35bc-4289-9732-91a9ab45ab67-maas-api-xks-ca" pod "ad250179-35bc-4289-9732-91a9ab45ab67" (UID: "ad250179-35bc-4289-9732-91a9ab45ab67"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:49.379708 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:49.379414    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ad250179-35bc-4289-9732-91a9ab45ab67-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ad250179-35bc-4289-9732-91a9ab45ab67-maas-api-openshift-ca\") pod \"ad250179-35bc-4289-9732-91a9ab45ab67\" (UID: \"ad250179-35bc-4289-9732-91a9ab45ab67\") "
Oct 30 19:33:49.380083 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:49.380057    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ad250179-35bc-4289-9732-91a9ab45ab67-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:49.380178 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:49.380089    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hm9pl\" (UniqueName: \"kubernetes.io/projected/ad250179-35bc-4289-9732-91a9ab45ab67-kube-api-access-hm9pl\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:49.380413 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:49.380382    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad250179-35bc-4289-9732-91a9ab45ab67-maas-api-openshift-ca" pod "ad250179-35bc-4289-9732-91a9ab45ab67" (UID: "ad250179-35bc-4289-9732-91a9ab45ab67"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:49.483449 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:49.483410    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ad250179-35bc-4289-9732-91a9ab45ab67-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:50.081280 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:50.081246    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3eb9ce78-524x8"
Oct 30 19:33:50.081475 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:50.081279    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3eb9ce78-524x8" event={"ID":"ad250179-35bc-4289-9732-91a9ab45ab67","Type":"ContainerDied","Data":"f28303a90cea3ed5b46ce21c2007a87e2c288994d566aa6df432df1f0b444775"}
Oct 30 19:33:50.081475 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:50.081330    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f28303a90cea3ed5b46ce21c2007a87e2c288994d566aa6df432df1f0b444775"
Oct 30 19:33:51.403634 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:51.403598    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7a76fa6b-75gxz"]
Oct 30 19:33:51.404216 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:51.404202    5907 cpu_manager.go:414] "containerMap: removing container" podUID="ad250179-35bc-4289-9732-91a9ab45ab67" containerName="revoke-keys"
Oct 30 19:33:51.404216 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:51.404217    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ad250179-35bc-4289-9732-91a9ab45ab67" containerName="revoke-keys"
Oct 30 19:33:51.404310 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:51.404302    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="ad250179-35bc-4289-9732-91a9ab45ab67" containerName="revoke-keys"
Oct 30 19:33:51.407819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:51.407797    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a76fa6b-75gxz"
Oct 30 19:33:51.410431 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:51.410407    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6e93fb18-a78b-458c-a4a9-c47d88f7ac4d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7a76fa6b-75gxz\" (UID: \"6e93fb18-a78b-458c-a4a9-c47d88f7ac4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a76fa6b-75gxz"
Oct 30 19:33:51.410588 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:51.410550    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6e93fb18-a78b-458c-a4a9-c47d88f7ac4d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7a76fa6b-75gxz\" (UID: \"6e93fb18-a78b-458c-a4a9-c47d88f7ac4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a76fa6b-75gxz"
Oct 30 19:33:51.410657 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:51.410594    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2klqr\" (UniqueName: \"kubernetes.io/projected/6e93fb18-a78b-458c-a4a9-c47d88f7ac4d-kube-api-access-2klqr\") pod \"maas-api-revoke-sub-7a76fa6b-75gxz\" (UID: \"6e93fb18-a78b-458c-a4a9-c47d88f7ac4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a76fa6b-75gxz"
Oct 30 19:33:51.427990 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:51.427962    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7a76fa6b-75gxz"]
Oct 30 19:33:51.436744 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:51.436716    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:33:51.436948 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:51.436797    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:33:51.511734 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:51.511701    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6e93fb18-a78b-458c-a4a9-c47d88f7ac4d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7a76fa6b-75gxz\" (UID: \"6e93fb18-a78b-458c-a4a9-c47d88f7ac4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a76fa6b-75gxz"
Oct 30 19:33:51.511931 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:51.511747    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2klqr\" (UniqueName: \"kubernetes.io/projected/6e93fb18-a78b-458c-a4a9-c47d88f7ac4d-kube-api-access-2klqr\") pod \"maas-api-revoke-sub-7a76fa6b-75gxz\" (UID: \"6e93fb18-a78b-458c-a4a9-c47d88f7ac4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a76fa6b-75gxz"
Oct 30 19:33:51.511931 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:51.511806    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6e93fb18-a78b-458c-a4a9-c47d88f7ac4d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7a76fa6b-75gxz\" (UID: \"6e93fb18-a78b-458c-a4a9-c47d88f7ac4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a76fa6b-75gxz"
Oct 30 19:33:51.512460 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:51.512441    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6e93fb18-a78b-458c-a4a9-c47d88f7ac4d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7a76fa6b-75gxz\" (UID: \"6e93fb18-a78b-458c-a4a9-c47d88f7ac4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a76fa6b-75gxz"
Oct 30 19:33:51.514300 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:51.514275    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6e93fb18-a78b-458c-a4a9-c47d88f7ac4d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7a76fa6b-75gxz\" (UID: \"6e93fb18-a78b-458c-a4a9-c47d88f7ac4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a76fa6b-75gxz"
Oct 30 19:33:51.533690 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:51.533667    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2klqr\" (UniqueName: \"kubernetes.io/projected/6e93fb18-a78b-458c-a4a9-c47d88f7ac4d-kube-api-access-2klqr\") pod \"maas-api-revoke-sub-7a76fa6b-75gxz\" (UID: \"6e93fb18-a78b-458c-a4a9-c47d88f7ac4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a76fa6b-75gxz"
Oct 30 19:33:51.703836 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:51.703735    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e8596f5d-6kbzh"]
Oct 30 19:33:51.708279 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:51.708255    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e8596f5d-6kbzh"
Oct 30 19:33:51.713749 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:51.713726    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ddf77163-3370-4a29-a119-abe7e1723841-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e8596f5d-6kbzh\" (UID: \"ddf77163-3370-4a29-a119-abe7e1723841\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e8596f5d-6kbzh"
Oct 30 19:33:51.713871 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:51.713790    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rzpr\" (UniqueName: \"kubernetes.io/projected/ddf77163-3370-4a29-a119-abe7e1723841-kube-api-access-7rzpr\") pod \"maas-api-revoke-sub-e8596f5d-6kbzh\" (UID: \"ddf77163-3370-4a29-a119-abe7e1723841\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e8596f5d-6kbzh"
Oct 30 19:33:51.713959 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:51.713936    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ddf77163-3370-4a29-a119-abe7e1723841-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e8596f5d-6kbzh\" (UID: \"ddf77163-3370-4a29-a119-abe7e1723841\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e8596f5d-6kbzh"
Oct 30 19:33:51.721526 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:51.721502    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a76fa6b-75gxz"
Oct 30 19:33:51.727973 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:51.727946    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e8596f5d-6kbzh"]
Oct 30 19:33:51.815644 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:51.815608    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rzpr\" (UniqueName: \"kubernetes.io/projected/ddf77163-3370-4a29-a119-abe7e1723841-kube-api-access-7rzpr\") pod \"maas-api-revoke-sub-e8596f5d-6kbzh\" (UID: \"ddf77163-3370-4a29-a119-abe7e1723841\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e8596f5d-6kbzh"
Oct 30 19:33:51.815830 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:51.815702    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ddf77163-3370-4a29-a119-abe7e1723841-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e8596f5d-6kbzh\" (UID: \"ddf77163-3370-4a29-a119-abe7e1723841\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e8596f5d-6kbzh"
Oct 30 19:33:51.815920 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:51.815879    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ddf77163-3370-4a29-a119-abe7e1723841-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e8596f5d-6kbzh\" (UID: \"ddf77163-3370-4a29-a119-abe7e1723841\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e8596f5d-6kbzh"
Oct 30 19:33:51.816463 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:51.816435    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ddf77163-3370-4a29-a119-abe7e1723841-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e8596f5d-6kbzh\" (UID: \"ddf77163-3370-4a29-a119-abe7e1723841\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e8596f5d-6kbzh"
Oct 30 19:33:51.818284 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:51.818261    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ddf77163-3370-4a29-a119-abe7e1723841-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e8596f5d-6kbzh\" (UID: \"ddf77163-3370-4a29-a119-abe7e1723841\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e8596f5d-6kbzh"
Oct 30 19:33:51.837639 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:51.837609    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rzpr\" (UniqueName: \"kubernetes.io/projected/ddf77163-3370-4a29-a119-abe7e1723841-kube-api-access-7rzpr\") pod \"maas-api-revoke-sub-e8596f5d-6kbzh\" (UID: \"ddf77163-3370-4a29-a119-abe7e1723841\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e8596f5d-6kbzh"
Oct 30 19:33:52.023277 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:52.023195    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e8596f5d-6kbzh"
Oct 30 19:33:52.043957 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:52.043928    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7a76fa6b-75gxz"]
Oct 30 19:33:52.044639 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:33:52.044610    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e93fb18_a78b_458c_a4a9_c47d88f7ac4d.slice/crio-4a68105cbc73f6e753be43b2859722aea0c7baf71704f723b6835d560fe01560 WatchSource:0}: Error finding container 4a68105cbc73f6e753be43b2859722aea0c7baf71704f723b6835d560fe01560: Status 404 returned error can't find the container with id 4a68105cbc73f6e753be43b2859722aea0c7baf71704f723b6835d560fe01560
Oct 30 19:33:52.103006 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:52.102889    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a76fa6b-75gxz" event={"ID":"6e93fb18-a78b-458c-a4a9-c47d88f7ac4d","Type":"ContainerStarted","Data":"4a68105cbc73f6e753be43b2859722aea0c7baf71704f723b6835d560fe01560"}
Oct 30 19:33:52.378533 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:52.378509    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e8596f5d-6kbzh"]
Oct 30 19:33:52.379849 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:33:52.379827    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddf77163_3370_4a29_a119_abe7e1723841.slice/crio-3bdd00671c4cb82e178690ca389ccffda78ad6d0aae0e6f882b784f886d7cd91 WatchSource:0}: Error finding container 3bdd00671c4cb82e178690ca389ccffda78ad6d0aae0e6f882b784f886d7cd91: Status 404 returned error can't find the container with id 3bdd00671c4cb82e178690ca389ccffda78ad6d0aae0e6f882b784f886d7cd91
Oct 30 19:33:53.111460 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:53.111419    5907 generic.go:356] "Generic (PLEG): container finished" podID="6e93fb18-a78b-458c-a4a9-c47d88f7ac4d" containerID="55986730e1b950b0060a304170d843ed7e7a9c3e95cd54a0f10a59f113df1649" exitCode=0
Oct 30 19:33:53.112234 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:53.111521    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a76fa6b-75gxz" event={"ID":"6e93fb18-a78b-458c-a4a9-c47d88f7ac4d","Type":"ContainerDied","Data":"55986730e1b950b0060a304170d843ed7e7a9c3e95cd54a0f10a59f113df1649"}
Oct 30 19:33:53.113486 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:53.113453    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e8596f5d-6kbzh" event={"ID":"ddf77163-3370-4a29-a119-abe7e1723841","Type":"ContainerStarted","Data":"b2d9470e5dc990dfa05440af3ae53f9b834a47afe7f770ec4f49385de31e19c7"}
Oct 30 19:33:53.113644 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:53.113524    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e8596f5d-6kbzh" event={"ID":"ddf77163-3370-4a29-a119-abe7e1723841","Type":"ContainerStarted","Data":"3bdd00671c4cb82e178690ca389ccffda78ad6d0aae0e6f882b784f886d7cd91"}
Oct 30 19:33:53.197218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:53.197157    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e8596f5d-6kbzh" podStartSLOduration=2.197143812 podStartE2EDuration="2.197143812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:33:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:33:53.193835274 +0000 UTC m=+745.036767212" watchObservedRunningTime="2026-10-30 19:33:53.197143812 +0000 UTC m=+745.040075768"
Oct 30 19:33:54.018376 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.018347    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"]
Oct 30 19:33:54.022644 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.022624    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"
Oct 30 19:33:54.043896 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.043812    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-pdg78\" (UID: \"bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"
Oct 30 19:33:54.043896 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.043886    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-pdg78\" (UID: \"bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"
Oct 30 19:33:54.044199 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.044040    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-pdg78\" (UID: \"bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"
Oct 30 19:33:54.044199 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.044106    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-pdg78\" (UID: \"bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"
Oct 30 19:33:54.044199 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.044159    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-pdg78\" (UID: \"bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"
Oct 30 19:33:54.044392 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.044197    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lr6pg\" (UniqueName: \"kubernetes.io/projected/bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3-kube-api-access-lr6pg\") pod \"maas-default-gateway-openshift-default-5bc884dd87-pdg78\" (UID: \"bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"
Oct 30 19:33:54.044392 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.044262    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-pdg78\" (UID: \"bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"
Oct 30 19:33:54.044392 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.044344    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-pdg78\" (UID: \"bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"
Oct 30 19:33:54.044573 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.044455    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-pdg78\" (UID: \"bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"
Oct 30 19:33:54.056779 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.056746    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"]
Oct 30 19:33:54.126003 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.125964    5907 generic.go:356] "Generic (PLEG): container finished" podID="ddf77163-3370-4a29-a119-abe7e1723841" containerID="b2d9470e5dc990dfa05440af3ae53f9b834a47afe7f770ec4f49385de31e19c7" exitCode=0
Oct 30 19:33:54.126485 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.126059    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e8596f5d-6kbzh" event={"ID":"ddf77163-3370-4a29-a119-abe7e1723841","Type":"ContainerDied","Data":"b2d9470e5dc990dfa05440af3ae53f9b834a47afe7f770ec4f49385de31e19c7"}
Oct 30 19:33:54.145726 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.145692    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-pdg78\" (UID: \"bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"
Oct 30 19:33:54.145988 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.145736    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-pdg78\" (UID: \"bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"
Oct 30 19:33:54.145988 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.145764    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-pdg78\" (UID: \"bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"
Oct 30 19:33:54.145988 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.145829    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-pdg78\" (UID: \"bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"
Oct 30 19:33:54.145988 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.145901    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-pdg78\" (UID: \"bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"
Oct 30 19:33:54.145988 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.145948    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-pdg78\" (UID: \"bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"
Oct 30 19:33:54.145988 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.145987    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lr6pg\" (UniqueName: \"kubernetes.io/projected/bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3-kube-api-access-lr6pg\") pod \"maas-default-gateway-openshift-default-5bc884dd87-pdg78\" (UID: \"bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"
Oct 30 19:33:54.146269 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.146050    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-pdg78\" (UID: \"bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"
Oct 30 19:33:54.146269 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.146138    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-pdg78\" (UID: \"bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"
Oct 30 19:33:54.147043 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.146276    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-pdg78\" (UID: \"bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"
Oct 30 19:33:54.147043 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.146334    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-pdg78\" (UID: \"bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"
Oct 30 19:33:54.147043 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.146581    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-pdg78\" (UID: \"bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"
Oct 30 19:33:54.147043 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.146785    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-pdg78\" (UID: \"bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"
Oct 30 19:33:54.147043 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.146800    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-pdg78\" (UID: \"bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"
Oct 30 19:33:54.148561 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.148537    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-pdg78\" (UID: \"bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"
Oct 30 19:33:54.148696 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.148676    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-pdg78\" (UID: \"bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"
Oct 30 19:33:54.166958 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.166923    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lr6pg\" (UniqueName: \"kubernetes.io/projected/bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3-kube-api-access-lr6pg\") pod \"maas-default-gateway-openshift-default-5bc884dd87-pdg78\" (UID: \"bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"
Oct 30 19:33:54.167089 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.166978    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-pdg78\" (UID: \"bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"
Oct 30 19:33:54.314911 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.314886    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a76fa6b-75gxz"
Oct 30 19:33:54.338496 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.338470    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"
Oct 30 19:33:54.348719 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.348692    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6e93fb18-a78b-458c-a4a9-c47d88f7ac4d-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6e93fb18-a78b-458c-a4a9-c47d88f7ac4d-maas-api-openshift-ca\") pod \"6e93fb18-a78b-458c-a4a9-c47d88f7ac4d\" (UID: \"6e93fb18-a78b-458c-a4a9-c47d88f7ac4d\") "
Oct 30 19:33:54.348854 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.348778    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6e93fb18-a78b-458c-a4a9-c47d88f7ac4d-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6e93fb18-a78b-458c-a4a9-c47d88f7ac4d-maas-api-xks-ca\") pod \"6e93fb18-a78b-458c-a4a9-c47d88f7ac4d\" (UID: \"6e93fb18-a78b-458c-a4a9-c47d88f7ac4d\") "
Oct 30 19:33:54.348967 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.348902    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6e93fb18-a78b-458c-a4a9-c47d88f7ac4d-kube-api-access-2klqr\" (UniqueName: \"kubernetes.io/projected/6e93fb18-a78b-458c-a4a9-c47d88f7ac4d-kube-api-access-2klqr\") pod \"6e93fb18-a78b-458c-a4a9-c47d88f7ac4d\" (UID: \"6e93fb18-a78b-458c-a4a9-c47d88f7ac4d\") "
Oct 30 19:33:54.349107 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.349085    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e93fb18-a78b-458c-a4a9-c47d88f7ac4d-maas-api-openshift-ca" pod "6e93fb18-a78b-458c-a4a9-c47d88f7ac4d" (UID: "6e93fb18-a78b-458c-a4a9-c47d88f7ac4d"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:54.349400 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.349375    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6e93fb18-a78b-458c-a4a9-c47d88f7ac4d-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:54.351645 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.351616    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e93fb18-a78b-458c-a4a9-c47d88f7ac4d-maas-api-xks-ca" pod "6e93fb18-a78b-458c-a4a9-c47d88f7ac4d" (UID: "6e93fb18-a78b-458c-a4a9-c47d88f7ac4d"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:54.352154 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.352132    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e93fb18-a78b-458c-a4a9-c47d88f7ac4d-kube-api-access-2klqr" pod "6e93fb18-a78b-458c-a4a9-c47d88f7ac4d" (UID: "6e93fb18-a78b-458c-a4a9-c47d88f7ac4d"). InnerVolumeSpecName "kube-api-access-2klqr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:54.450911 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.450879    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6e93fb18-a78b-458c-a4a9-c47d88f7ac4d-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:54.450911 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.450911    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2klqr\" (UniqueName: \"kubernetes.io/projected/6e93fb18-a78b-458c-a4a9-c47d88f7ac4d-kube-api-access-2klqr\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:54.692443 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.692410    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"]
Oct 30 19:33:54.693325 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:33:54.693294    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf9df589_b7ee_40ae_acdd_c0c1fc5eccf3.slice/crio-833dedf5778f8856b2a9d7eb0ef0a1ce4265250a5cfe2e9f78a41555e49edc1e WatchSource:0}: Error finding container 833dedf5778f8856b2a9d7eb0ef0a1ce4265250a5cfe2e9f78a41555e49edc1e: Status 404 returned error can't find the container with id 833dedf5778f8856b2a9d7eb0ef0a1ce4265250a5cfe2e9f78a41555e49edc1e
Oct 30 19:33:54.695984 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.695947    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:33:54.696076 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.696015    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:33:54.696076 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:54.696047    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:33:55.137786 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:55.137734    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a76fa6b-75gxz" event={"ID":"6e93fb18-a78b-458c-a4a9-c47d88f7ac4d","Type":"ContainerDied","Data":"4a68105cbc73f6e753be43b2859722aea0c7baf71704f723b6835d560fe01560"}
Oct 30 19:33:55.138333 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:55.137814    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4a68105cbc73f6e753be43b2859722aea0c7baf71704f723b6835d560fe01560"
Oct 30 19:33:55.140154 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:55.140127    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a76fa6b-75gxz"
Oct 30 19:33:55.141011 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:55.140983    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78" event={"ID":"bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3","Type":"ContainerStarted","Data":"bdcf03f675241c3c2bf1311430129c5252feb885e3b8946699e4fb28c7b2ba90"}
Oct 30 19:33:55.141141 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:55.141040    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78" event={"ID":"bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3","Type":"ContainerStarted","Data":"833dedf5778f8856b2a9d7eb0ef0a1ce4265250a5cfe2e9f78a41555e49edc1e"}
Oct 30 19:33:55.341329 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:55.341251    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"
Oct 30 19:33:55.343113 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:55.343076    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-pdg78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 19:33:55.343245 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:55.343158    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78" podUID="bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 19:33:55.349260 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:55.349238    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e8596f5d-6kbzh"
Oct 30 19:33:55.392444 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:55.392338    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78" podStartSLOduration=2.392323963 podStartE2EDuration="2.392323963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:33:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:33:55.199451675 +0000 UTC m=+747.042383613" watchObservedRunningTime="2026-10-30 19:33:55.392323963 +0000 UTC m=+747.235255903"
Oct 30 19:33:55.464649 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:55.464613    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ddf77163-3370-4a29-a119-abe7e1723841-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ddf77163-3370-4a29-a119-abe7e1723841-maas-api-xks-ca\") pod \"ddf77163-3370-4a29-a119-abe7e1723841\" (UID: \"ddf77163-3370-4a29-a119-abe7e1723841\") "
Oct 30 19:33:55.464804 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:55.464789    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ddf77163-3370-4a29-a119-abe7e1723841-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ddf77163-3370-4a29-a119-abe7e1723841-maas-api-openshift-ca\") pod \"ddf77163-3370-4a29-a119-abe7e1723841\" (UID: \"ddf77163-3370-4a29-a119-abe7e1723841\") "
Oct 30 19:33:55.464874 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:55.464851    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ddf77163-3370-4a29-a119-abe7e1723841-kube-api-access-7rzpr\" (UniqueName: \"kubernetes.io/projected/ddf77163-3370-4a29-a119-abe7e1723841-kube-api-access-7rzpr\") pod \"ddf77163-3370-4a29-a119-abe7e1723841\" (UID: \"ddf77163-3370-4a29-a119-abe7e1723841\") "
Oct 30 19:33:55.465170 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:55.465143    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddf77163-3370-4a29-a119-abe7e1723841-maas-api-openshift-ca" pod "ddf77163-3370-4a29-a119-abe7e1723841" (UID: "ddf77163-3370-4a29-a119-abe7e1723841"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:55.465350 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:55.465288    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ddf77163-3370-4a29-a119-abe7e1723841-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:55.467034 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:55.467014    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddf77163-3370-4a29-a119-abe7e1723841-maas-api-xks-ca" pod "ddf77163-3370-4a29-a119-abe7e1723841" (UID: "ddf77163-3370-4a29-a119-abe7e1723841"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:55.467122 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:55.467096    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddf77163-3370-4a29-a119-abe7e1723841-kube-api-access-7rzpr" pod "ddf77163-3370-4a29-a119-abe7e1723841" (UID: "ddf77163-3370-4a29-a119-abe7e1723841"). InnerVolumeSpecName "kube-api-access-7rzpr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:55.566321 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:55.566288    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rzpr\" (UniqueName: \"kubernetes.io/projected/ddf77163-3370-4a29-a119-abe7e1723841-kube-api-access-7rzpr\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:55.566321 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:55.566317    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ddf77163-3370-4a29-a119-abe7e1723841-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:33:56.156652 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:56.156466    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e8596f5d-6kbzh" event={"ID":"ddf77163-3370-4a29-a119-abe7e1723841","Type":"ContainerDied","Data":"3bdd00671c4cb82e178690ca389ccffda78ad6d0aae0e6f882b784f886d7cd91"}
Oct 30 19:33:56.156652 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:56.156540    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e8596f5d-6kbzh"
Oct 30 19:33:56.157125 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:56.156978    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3bdd00671c4cb82e178690ca389ccffda78ad6d0aae0e6f882b784f886d7cd91"
Oct 30 19:33:56.339258 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:56.339221    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-pdg78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 19:33:56.339444 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:56.339279    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78" podUID="bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 19:33:56.553156 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:56.553078    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7a9bf31c-knzts"]
Oct 30 19:33:56.553717 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:56.553700    5907 cpu_manager.go:414] "containerMap: removing container" podUID="6e93fb18-a78b-458c-a4a9-c47d88f7ac4d" containerName="revoke-keys"
Oct 30 19:33:56.553789 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:56.553717    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6e93fb18-a78b-458c-a4a9-c47d88f7ac4d" containerName="revoke-keys"
Oct 30 19:33:56.553789 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:56.553758    5907 cpu_manager.go:414] "containerMap: removing container" podUID="ddf77163-3370-4a29-a119-abe7e1723841" containerName="revoke-keys"
Oct 30 19:33:56.553789 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:56.553763    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ddf77163-3370-4a29-a119-abe7e1723841" containerName="revoke-keys"
Oct 30 19:33:56.553914 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:56.553850    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="ddf77163-3370-4a29-a119-abe7e1723841" containerName="revoke-keys"
Oct 30 19:33:56.553914 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:56.553897    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="6e93fb18-a78b-458c-a4a9-c47d88f7ac4d" containerName="revoke-keys"
Oct 30 19:33:56.557680 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:56.557647    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a9bf31c-knzts"
Oct 30 19:33:56.575511 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:56.575483    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:33:56.575650 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:56.575559    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:33:56.579987 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:56.578593    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7a9bf31c-knzts\" (UID: \"cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a9bf31c-knzts"
Oct 30 19:33:56.579987 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:56.578642    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7a9bf31c-knzts\" (UID: \"cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a9bf31c-knzts"
Oct 30 19:33:56.579987 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:56.578713    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snfjj\" (UniqueName: \"kubernetes.io/projected/cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927-kube-api-access-snfjj\") pod \"maas-api-revoke-sub-7a9bf31c-knzts\" (UID: \"cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a9bf31c-knzts"
Oct 30 19:33:56.579987 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:56.579753    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7a9bf31c-knzts"]
Oct 30 19:33:56.679412 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:56.679375    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snfjj\" (UniqueName: \"kubernetes.io/projected/cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927-kube-api-access-snfjj\") pod \"maas-api-revoke-sub-7a9bf31c-knzts\" (UID: \"cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a9bf31c-knzts"
Oct 30 19:33:56.679622 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:56.679487    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7a9bf31c-knzts\" (UID: \"cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a9bf31c-knzts"
Oct 30 19:33:56.679622 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:56.679519    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7a9bf31c-knzts\" (UID: \"cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a9bf31c-knzts"
Oct 30 19:33:56.680227 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:56.680200    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7a9bf31c-knzts\" (UID: \"cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a9bf31c-knzts"
Oct 30 19:33:56.681999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:56.681973    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7a9bf31c-knzts\" (UID: \"cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a9bf31c-knzts"
Oct 30 19:33:56.699951 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:56.699931    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-snfjj\" (UniqueName: \"kubernetes.io/projected/cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927-kube-api-access-snfjj\") pod \"maas-api-revoke-sub-7a9bf31c-knzts\" (UID: \"cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a9bf31c-knzts"
Oct 30 19:33:56.872247 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:56.872222    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a9bf31c-knzts"
Oct 30 19:33:57.204468 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:57.204435    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7a9bf31c-knzts"]
Oct 30 19:33:57.204940 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:33:57.204813    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfb8f2ab_9cc6_4a0b_9ca2_20e5d34b2927.slice/crio-845a9235d81c5a64b7d76ac782406de2146ad5e9d41575b19b2772a474623be8 WatchSource:0}: Error finding container 845a9235d81c5a64b7d76ac782406de2146ad5e9d41575b19b2772a474623be8: Status 404 returned error can't find the container with id 845a9235d81c5a64b7d76ac782406de2146ad5e9d41575b19b2772a474623be8
Oct 30 19:33:57.340310 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:57.340273    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-pdg78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 19:33:57.340480 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:57.340327    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78" podUID="bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 19:33:58.170992 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:58.170950    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a9bf31c-knzts" event={"ID":"cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927","Type":"ContainerStarted","Data":"4d821bd0714776f533c0ef2a043171eeed181d381462c12e68e76945752d94ee"}
Oct 30 19:33:58.171195 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:58.171003    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a9bf31c-knzts" event={"ID":"cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927","Type":"ContainerStarted","Data":"845a9235d81c5a64b7d76ac782406de2146ad5e9d41575b19b2772a474623be8"}
Oct 30 19:33:58.340461 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:58.340417    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-pdg78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 19:33:58.340895 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:58.340480    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78" podUID="bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 19:33:59.179294 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:59.179251    5907 generic.go:356] "Generic (PLEG): container finished" podID="cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927" containerID="4d821bd0714776f533c0ef2a043171eeed181d381462c12e68e76945752d94ee" exitCode=0
Oct 30 19:33:59.179486 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:59.179303    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a9bf31c-knzts" event={"ID":"cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927","Type":"ContainerDied","Data":"4d821bd0714776f533c0ef2a043171eeed181d381462c12e68e76945752d94ee"}
Oct 30 19:33:59.339404 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:59.339362    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-pdg78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 19:33:59.339613 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:59.339440    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78" podUID="bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 19:33:59.836887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:59.836449    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"]
Oct 30 19:33:59.901559 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:59.897488    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"]
Oct 30 19:33:59.901559 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:59.897627    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"
Oct 30 19:33:59.944585 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:59.944545    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zqvn\" (UniqueName: \"kubernetes.io/projected/283578cb-7837-4d4a-b91f-dee1d0d74322-kube-api-access-4zqvn\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"
Oct 30 19:33:59.944762 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:59.944693    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/283578cb-7837-4d4a-b91f-dee1d0d74322-credential-socket\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"
Oct 30 19:33:59.944762 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:59.944731    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/283578cb-7837-4d4a-b91f-dee1d0d74322-workload-certs\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"
Oct 30 19:33:59.944942 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:59.944812    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/283578cb-7837-4d4a-b91f-dee1d0d74322-istio-envoy\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"
Oct 30 19:33:59.944942 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:59.944931    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/283578cb-7837-4d4a-b91f-dee1d0d74322-istio-token\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"
Oct 30 19:33:59.945052 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:59.944988    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/283578cb-7837-4d4a-b91f-dee1d0d74322-workload-socket\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"
Oct 30 19:33:59.945052 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:59.945008    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/283578cb-7837-4d4a-b91f-dee1d0d74322-istio-data\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"
Oct 30 19:33:59.945052 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:59.945029    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/283578cb-7837-4d4a-b91f-dee1d0d74322-istio-podinfo\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"
Oct 30 19:33:59.945199 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:33:59.945091    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/283578cb-7837-4d4a-b91f-dee1d0d74322-istiod-ca-cert\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"
Oct 30 19:34:00.046698 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:00.046651    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/283578cb-7837-4d4a-b91f-dee1d0d74322-istio-token\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"
Oct 30 19:34:00.046698 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:00.046696    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/283578cb-7837-4d4a-b91f-dee1d0d74322-workload-socket\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"
Oct 30 19:34:00.046974 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:00.046715    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/283578cb-7837-4d4a-b91f-dee1d0d74322-istio-data\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"
Oct 30 19:34:00.046974 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:00.046737    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/283578cb-7837-4d4a-b91f-dee1d0d74322-istio-podinfo\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"
Oct 30 19:34:00.046974 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:00.046771    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/283578cb-7837-4d4a-b91f-dee1d0d74322-istiod-ca-cert\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"
Oct 30 19:34:00.046974 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:00.046851    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zqvn\" (UniqueName: \"kubernetes.io/projected/283578cb-7837-4d4a-b91f-dee1d0d74322-kube-api-access-4zqvn\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"
Oct 30 19:34:00.046974 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:00.046948    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/283578cb-7837-4d4a-b91f-dee1d0d74322-credential-socket\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"
Oct 30 19:34:00.047263 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:00.046981    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/283578cb-7837-4d4a-b91f-dee1d0d74322-workload-certs\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"
Oct 30 19:34:00.047263 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:00.047018    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/283578cb-7837-4d4a-b91f-dee1d0d74322-istio-envoy\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"
Oct 30 19:34:00.047263 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:00.047169    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/283578cb-7837-4d4a-b91f-dee1d0d74322-workload-socket\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"
Oct 30 19:34:00.047263 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:00.047214    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/283578cb-7837-4d4a-b91f-dee1d0d74322-istio-data\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"
Oct 30 19:34:00.047532 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:00.047276    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/283578cb-7837-4d4a-b91f-dee1d0d74322-credential-socket\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"
Oct 30 19:34:00.047532 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:00.047397    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/283578cb-7837-4d4a-b91f-dee1d0d74322-workload-certs\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"
Oct 30 19:34:00.047706 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:00.047684    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/283578cb-7837-4d4a-b91f-dee1d0d74322-istiod-ca-cert\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"
Oct 30 19:34:00.049374 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:00.049346    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/283578cb-7837-4d4a-b91f-dee1d0d74322-istio-envoy\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"
Oct 30 19:34:00.049708 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:00.049686    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/283578cb-7837-4d4a-b91f-dee1d0d74322-istio-podinfo\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"
Oct 30 19:34:00.067849 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:00.067821    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/283578cb-7837-4d4a-b91f-dee1d0d74322-istio-token\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"
Oct 30 19:34:00.068118 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:00.068097    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zqvn\" (UniqueName: \"kubernetes.io/projected/283578cb-7837-4d4a-b91f-dee1d0d74322-kube-api-access-4zqvn\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"
Oct 30 19:34:00.242465 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:00.242425    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"
Oct 30 19:34:00.382472 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:00.382296    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-pdg78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 19:34:00.382472 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:00.382362    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78" podUID="bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 19:34:00.466344 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:00.466319    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a9bf31c-knzts"
Oct 30 19:34:00.572925 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:00.572897    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"]
Oct 30 19:34:00.573329 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:34:00.573308    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod283578cb_7837_4d4a_b91f_dee1d0d74322.slice/crio-d5b1084784c349828c0b52d257a4d70720882bb38f885031a0ae1f34ef408019 WatchSource:0}: Error finding container d5b1084784c349828c0b52d257a4d70720882bb38f885031a0ae1f34ef408019: Status 404 returned error can't find the container with id d5b1084784c349828c0b52d257a4d70720882bb38f885031a0ae1f34ef408019
Oct 30 19:34:00.575796 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:00.575767    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:34:00.575890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:00.575828    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:34:00.575890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:00.575881    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:34:00.586460 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:00.586437    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927-kube-api-access-snfjj\" (UniqueName: \"kubernetes.io/projected/cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927-kube-api-access-snfjj\") pod \"cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927\" (UID: \"cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927\") "
Oct 30 19:34:00.586551 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:00.586493    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927-maas-api-xks-ca\") pod \"cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927\" (UID: \"cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927\") "
Oct 30 19:34:00.586628 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:00.586616    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927-maas-api-openshift-ca\") pod \"cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927\" (UID: \"cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927\") "
Oct 30 19:34:00.587300 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:00.587212    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927-maas-api-openshift-ca" pod "cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927" (UID: "cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:34:00.589033 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:00.588981    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927-maas-api-xks-ca" pod "cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927" (UID: "cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:34:00.589107 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:00.589012    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927-kube-api-access-snfjj" pod "cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927" (UID: "cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927"). InnerVolumeSpecName "kube-api-access-snfjj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:34:00.693356 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:00.688957    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-snfjj\" (UniqueName: \"kubernetes.io/projected/cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927-kube-api-access-snfjj\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:34:00.693356 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:00.688992    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:34:00.693356 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:00.689007    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:34:01.195011 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:01.194975    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a9bf31c-knzts"
Oct 30 19:34:01.195011 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:01.194989    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a9bf31c-knzts" event={"ID":"cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927","Type":"ContainerDied","Data":"845a9235d81c5a64b7d76ac782406de2146ad5e9d41575b19b2772a474623be8"}
Oct 30 19:34:01.195517 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:01.195050    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="845a9235d81c5a64b7d76ac782406de2146ad5e9d41575b19b2772a474623be8"
Oct 30 19:34:01.196776 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:01.196748    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb" event={"ID":"283578cb-7837-4d4a-b91f-dee1d0d74322","Type":"ContainerStarted","Data":"555010fb5775749893e35e327ebf73af2bc1ac9bcd8b0e180889649887577e51"}
Oct 30 19:34:01.196931 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:01.196804    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb" event={"ID":"283578cb-7837-4d4a-b91f-dee1d0d74322","Type":"ContainerStarted","Data":"d5b1084784c349828c0b52d257a4d70720882bb38f885031a0ae1f34ef408019"}
Oct 30 19:34:01.242907 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:01.242854    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"
Oct 30 19:34:01.244974 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:01.244948    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 19:34:01.245087 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:01.244994    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb" podUID="283578cb-7837-4d4a-b91f-dee1d0d74322" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 19:34:01.266483 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:01.266429    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb" podStartSLOduration=2.266415327 podStartE2EDuration="2.266415327s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:33:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:34:01.254717752 +0000 UTC m=+753.097649690" watchObservedRunningTime="2026-10-30 19:34:01.266415327 +0000 UTC m=+753.109347303"
Oct 30 19:34:01.339688 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:01.339606    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-pdg78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 19:34:01.339834 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:01.339677    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78" podUID="bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 19:34:02.244010 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:02.243977    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 19:34:02.244436 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:02.244035    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb" podUID="283578cb-7837-4d4a-b91f-dee1d0d74322" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 19:34:02.339557 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:02.339519    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-pdg78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 19:34:02.339814 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:02.339786    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78" podUID="bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 19:34:03.243455 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:03.243414    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 19:34:03.243622 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:03.243477    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb" podUID="283578cb-7837-4d4a-b91f-dee1d0d74322" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 19:34:03.393746 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:03.393709    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"
Oct 30 19:34:03.394693 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:03.394669    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"
Oct 30 19:34:03.395433 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:03.395416    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78"
Oct 30 19:34:04.243618 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:04.243579    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 19:34:04.243807 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:04.243639    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb" podUID="283578cb-7837-4d4a-b91f-dee1d0d74322" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 19:34:05.243847 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:05.243808    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 19:34:05.244293 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:05.243903    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb" podUID="283578cb-7837-4d4a-b91f-dee1d0d74322" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 19:34:05.936875 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:05.936829    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6753d129-czr6b"]
Oct 30 19:34:05.937849 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:05.937827    5907 cpu_manager.go:414] "containerMap: removing container" podUID="cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927" containerName="revoke-keys"
Oct 30 19:34:05.937948 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:05.937873    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927" containerName="revoke-keys"
Oct 30 19:34:05.938093 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:05.938079    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927" containerName="revoke-keys"
Oct 30 19:34:05.943432 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:05.943412    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6753d129-czr6b"
Oct 30 19:34:05.957510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:05.957474    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6753d129-czr6b"]
Oct 30 19:34:05.966423 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:05.966397    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:34:05.966526 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:05.966516    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:34:06.082692 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:06.082657    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx9d9\" (UniqueName: \"kubernetes.io/projected/3821123c-0cb7-40dc-989c-c0333c6d1e7b-kube-api-access-xx9d9\") pod \"maas-api-revoke-sub-6753d129-czr6b\" (UID: \"3821123c-0cb7-40dc-989c-c0333c6d1e7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6753d129-czr6b"
Oct 30 19:34:06.082900 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:06.082731    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3821123c-0cb7-40dc-989c-c0333c6d1e7b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6753d129-czr6b\" (UID: \"3821123c-0cb7-40dc-989c-c0333c6d1e7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6753d129-czr6b"
Oct 30 19:34:06.082951 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:06.082931    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3821123c-0cb7-40dc-989c-c0333c6d1e7b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6753d129-czr6b\" (UID: \"3821123c-0cb7-40dc-989c-c0333c6d1e7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6753d129-czr6b"
Oct 30 19:34:06.183909 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:06.183856    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3821123c-0cb7-40dc-989c-c0333c6d1e7b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6753d129-czr6b\" (UID: \"3821123c-0cb7-40dc-989c-c0333c6d1e7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6753d129-czr6b"
Oct 30 19:34:06.184097 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:06.184002    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3821123c-0cb7-40dc-989c-c0333c6d1e7b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6753d129-czr6b\" (UID: \"3821123c-0cb7-40dc-989c-c0333c6d1e7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6753d129-czr6b"
Oct 30 19:34:06.184097 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:06.184055    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx9d9\" (UniqueName: \"kubernetes.io/projected/3821123c-0cb7-40dc-989c-c0333c6d1e7b-kube-api-access-xx9d9\") pod \"maas-api-revoke-sub-6753d129-czr6b\" (UID: \"3821123c-0cb7-40dc-989c-c0333c6d1e7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6753d129-czr6b"
Oct 30 19:34:06.184565 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:06.184536    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3821123c-0cb7-40dc-989c-c0333c6d1e7b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6753d129-czr6b\" (UID: \"3821123c-0cb7-40dc-989c-c0333c6d1e7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6753d129-czr6b"
Oct 30 19:34:06.186607 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:06.186583    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3821123c-0cb7-40dc-989c-c0333c6d1e7b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6753d129-czr6b\" (UID: \"3821123c-0cb7-40dc-989c-c0333c6d1e7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6753d129-czr6b"
Oct 30 19:34:06.224737 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:06.224629    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx9d9\" (UniqueName: \"kubernetes.io/projected/3821123c-0cb7-40dc-989c-c0333c6d1e7b-kube-api-access-xx9d9\") pod \"maas-api-revoke-sub-6753d129-czr6b\" (UID: \"3821123c-0cb7-40dc-989c-c0333c6d1e7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6753d129-czr6b"
Oct 30 19:34:06.243266 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:06.243234    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 19:34:06.243441 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:06.243293    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb" podUID="283578cb-7837-4d4a-b91f-dee1d0d74322" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 19:34:06.257921 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:06.257892    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6753d129-czr6b"
Oct 30 19:34:06.621940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:06.609486    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6753d129-czr6b"]
Oct 30 19:34:07.247503 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:07.247456    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 19:34:07.247811 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:07.247757    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb" podUID="283578cb-7837-4d4a-b91f-dee1d0d74322" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 19:34:07.302716 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:07.302635    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6753d129-czr6b" event={"ID":"3821123c-0cb7-40dc-989c-c0333c6d1e7b","Type":"ContainerStarted","Data":"7c31a903569ba943ef4ef5cf7983a096619e951ea31e4d1ad32087c7f6712460"}
Oct 30 19:34:08.371989 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:08.371951    5907 generic.go:356] "Generic (PLEG): container finished" podID="3821123c-0cb7-40dc-989c-c0333c6d1e7b" containerID="5dd8d3cf391195e86379f1e00952a0d12250f9a88d0376ba87b36de439c44870" exitCode=0
Oct 30 19:34:08.373359 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:08.373325    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6753d129-czr6b" event={"ID":"3821123c-0cb7-40dc-989c-c0333c6d1e7b","Type":"ContainerDied","Data":"5dd8d3cf391195e86379f1e00952a0d12250f9a88d0376ba87b36de439c44870"}
Oct 30 19:34:09.259295 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:09.259250    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:34:09.259555 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:09.259330    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb" podUID="283578cb-7837-4d4a-b91f-dee1d0d74322" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:34:09.302752 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:09.302718    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"
Oct 30 19:34:09.303291 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:09.303264    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"
Oct 30 19:34:09.304715 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:09.304689    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"
Oct 30 19:34:09.565523 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:09.565498    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6753d129-czr6b"
Oct 30 19:34:09.656956 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:09.656929    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3821123c-0cb7-40dc-989c-c0333c6d1e7b-kube-api-access-xx9d9\" (UniqueName: \"kubernetes.io/projected/3821123c-0cb7-40dc-989c-c0333c6d1e7b-kube-api-access-xx9d9\") pod \"3821123c-0cb7-40dc-989c-c0333c6d1e7b\" (UID: \"3821123c-0cb7-40dc-989c-c0333c6d1e7b\") "
Oct 30 19:34:09.657135 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:09.657027    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3821123c-0cb7-40dc-989c-c0333c6d1e7b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3821123c-0cb7-40dc-989c-c0333c6d1e7b-maas-api-xks-ca\") pod \"3821123c-0cb7-40dc-989c-c0333c6d1e7b\" (UID: \"3821123c-0cb7-40dc-989c-c0333c6d1e7b\") "
Oct 30 19:34:09.657135 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:09.657095    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3821123c-0cb7-40dc-989c-c0333c6d1e7b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3821123c-0cb7-40dc-989c-c0333c6d1e7b-maas-api-openshift-ca\") pod \"3821123c-0cb7-40dc-989c-c0333c6d1e7b\" (UID: \"3821123c-0cb7-40dc-989c-c0333c6d1e7b\") "
Oct 30 19:34:09.657459 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:09.657436    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3821123c-0cb7-40dc-989c-c0333c6d1e7b-maas-api-openshift-ca" pod "3821123c-0cb7-40dc-989c-c0333c6d1e7b" (UID: "3821123c-0cb7-40dc-989c-c0333c6d1e7b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:34:09.657551 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:09.657536    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3821123c-0cb7-40dc-989c-c0333c6d1e7b-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:34:09.659310 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:09.659277    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3821123c-0cb7-40dc-989c-c0333c6d1e7b-maas-api-xks-ca" pod "3821123c-0cb7-40dc-989c-c0333c6d1e7b" (UID: "3821123c-0cb7-40dc-989c-c0333c6d1e7b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:34:09.659396 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:09.659324    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3821123c-0cb7-40dc-989c-c0333c6d1e7b-kube-api-access-xx9d9" pod "3821123c-0cb7-40dc-989c-c0333c6d1e7b" (UID: "3821123c-0cb7-40dc-989c-c0333c6d1e7b"). InnerVolumeSpecName "kube-api-access-xx9d9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:34:09.759161 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:09.759129    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xx9d9\" (UniqueName: \"kubernetes.io/projected/3821123c-0cb7-40dc-989c-c0333c6d1e7b-kube-api-access-xx9d9\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:34:09.759161 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:09.759156    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3821123c-0cb7-40dc-989c-c0333c6d1e7b-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:34:10.389628 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:10.389587    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6753d129-czr6b" event={"ID":"3821123c-0cb7-40dc-989c-c0333c6d1e7b","Type":"ContainerDied","Data":"7c31a903569ba943ef4ef5cf7983a096619e951ea31e4d1ad32087c7f6712460"}
Oct 30 19:34:10.389772 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:10.389656    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7c31a903569ba943ef4ef5cf7983a096619e951ea31e4d1ad32087c7f6712460"
Oct 30 19:34:10.389772 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:10.389609    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6753d129-czr6b"
Oct 30 19:34:11.386353 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:11.386320    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-24ad9216-vcvqp"]
Oct 30 19:34:11.386964 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:11.386948    5907 cpu_manager.go:414] "containerMap: removing container" podUID="3821123c-0cb7-40dc-989c-c0333c6d1e7b" containerName="revoke-keys"
Oct 30 19:34:11.386964 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:11.386964    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3821123c-0cb7-40dc-989c-c0333c6d1e7b" containerName="revoke-keys"
Oct 30 19:34:11.387101 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:11.387052    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="3821123c-0cb7-40dc-989c-c0333c6d1e7b" containerName="revoke-keys"
Oct 30 19:34:11.390825 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:11.390803    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-24ad9216-vcvqp"
Oct 30 19:34:11.406707 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:11.406687    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:34:11.420049 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:11.420023    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:34:11.425830 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:11.425803    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-24ad9216-vcvqp"]
Oct 30 19:34:11.480801 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:11.480770    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/49804909-841f-4a00-abe1-8316ee7aa58f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-24ad9216-vcvqp\" (UID: \"49804909-841f-4a00-abe1-8316ee7aa58f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-24ad9216-vcvqp"
Oct 30 19:34:11.480958 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:11.480833    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnfrg\" (UniqueName: \"kubernetes.io/projected/49804909-841f-4a00-abe1-8316ee7aa58f-kube-api-access-wnfrg\") pod \"maas-api-revoke-sub-24ad9216-vcvqp\" (UID: \"49804909-841f-4a00-abe1-8316ee7aa58f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-24ad9216-vcvqp"
Oct 30 19:34:11.480958 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:11.480933    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/49804909-841f-4a00-abe1-8316ee7aa58f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-24ad9216-vcvqp\" (UID: \"49804909-841f-4a00-abe1-8316ee7aa58f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-24ad9216-vcvqp"
Oct 30 19:34:11.589434 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:11.589396    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/49804909-841f-4a00-abe1-8316ee7aa58f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-24ad9216-vcvqp\" (UID: \"49804909-841f-4a00-abe1-8316ee7aa58f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-24ad9216-vcvqp"
Oct 30 19:34:11.589647 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:11.589507    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnfrg\" (UniqueName: \"kubernetes.io/projected/49804909-841f-4a00-abe1-8316ee7aa58f-kube-api-access-wnfrg\") pod \"maas-api-revoke-sub-24ad9216-vcvqp\" (UID: \"49804909-841f-4a00-abe1-8316ee7aa58f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-24ad9216-vcvqp"
Oct 30 19:34:11.589647 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:11.589553    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/49804909-841f-4a00-abe1-8316ee7aa58f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-24ad9216-vcvqp\" (UID: \"49804909-841f-4a00-abe1-8316ee7aa58f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-24ad9216-vcvqp"
Oct 30 19:34:11.592528 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:11.592493    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/49804909-841f-4a00-abe1-8316ee7aa58f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-24ad9216-vcvqp\" (UID: \"49804909-841f-4a00-abe1-8316ee7aa58f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-24ad9216-vcvqp"
Oct 30 19:34:11.592907 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:11.592678    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/49804909-841f-4a00-abe1-8316ee7aa58f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-24ad9216-vcvqp\" (UID: \"49804909-841f-4a00-abe1-8316ee7aa58f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-24ad9216-vcvqp"
Oct 30 19:34:11.613748 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:11.613724    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnfrg\" (UniqueName: \"kubernetes.io/projected/49804909-841f-4a00-abe1-8316ee7aa58f-kube-api-access-wnfrg\") pod \"maas-api-revoke-sub-24ad9216-vcvqp\" (UID: \"49804909-841f-4a00-abe1-8316ee7aa58f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-24ad9216-vcvqp"
Oct 30 19:34:11.705355 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:11.705276    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-24ad9216-vcvqp"
Oct 30 19:34:12.019445 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:34:12.019418    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49804909_841f_4a00_abe1_8316ee7aa58f.slice/crio-3df928758aac3890f85561f8a24c30060d886e8ea81c74c3e5ef2b74d8c40581 WatchSource:0}: Error finding container 3df928758aac3890f85561f8a24c30060d886e8ea81c74c3e5ef2b74d8c40581: Status 404 returned error can't find the container with id 3df928758aac3890f85561f8a24c30060d886e8ea81c74c3e5ef2b74d8c40581
Oct 30 19:34:12.020159 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:12.020130    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-24ad9216-vcvqp"]
Oct 30 19:34:12.405187 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:12.405150    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-24ad9216-vcvqp" event={"ID":"49804909-841f-4a00-abe1-8316ee7aa58f","Type":"ContainerStarted","Data":"3a534aef16c7cb76926db7453cf2f9722eccf03786e1daf4869bfc1c451308fa"}
Oct 30 19:34:12.405607 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:12.405205    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-24ad9216-vcvqp" event={"ID":"49804909-841f-4a00-abe1-8316ee7aa58f","Type":"ContainerStarted","Data":"3df928758aac3890f85561f8a24c30060d886e8ea81c74c3e5ef2b74d8c40581"}
Oct 30 19:34:12.451421 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:12.451327    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-24ad9216-vcvqp" podStartSLOduration=1.45131291 podStartE2EDuration="1.45131291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:34:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:34:12.447025855 +0000 UTC m=+764.289957794" watchObservedRunningTime="2026-10-30 19:34:12.45131291 +0000 UTC m=+764.294244848"
Oct 30 19:34:13.415321 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:13.415288    5907 generic.go:356] "Generic (PLEG): container finished" podID="49804909-841f-4a00-abe1-8316ee7aa58f" containerID="3a534aef16c7cb76926db7453cf2f9722eccf03786e1daf4869bfc1c451308fa" exitCode=0
Oct 30 19:34:13.415805 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:13.415367    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-24ad9216-vcvqp" event={"ID":"49804909-841f-4a00-abe1-8316ee7aa58f","Type":"ContainerDied","Data":"3a534aef16c7cb76926db7453cf2f9722eccf03786e1daf4869bfc1c451308fa"}
Oct 30 19:34:14.578275 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:14.578251    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-24ad9216-vcvqp"
Oct 30 19:34:14.642519 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:14.642478    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/49804909-841f-4a00-abe1-8316ee7aa58f-kube-api-access-wnfrg\" (UniqueName: \"kubernetes.io/projected/49804909-841f-4a00-abe1-8316ee7aa58f-kube-api-access-wnfrg\") pod \"49804909-841f-4a00-abe1-8316ee7aa58f\" (UID: \"49804909-841f-4a00-abe1-8316ee7aa58f\") "
Oct 30 19:34:14.642710 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:14.642684    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/49804909-841f-4a00-abe1-8316ee7aa58f-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/49804909-841f-4a00-abe1-8316ee7aa58f-maas-api-xks-ca\") pod \"49804909-841f-4a00-abe1-8316ee7aa58f\" (UID: \"49804909-841f-4a00-abe1-8316ee7aa58f\") "
Oct 30 19:34:14.642787 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:14.642742    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/49804909-841f-4a00-abe1-8316ee7aa58f-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/49804909-841f-4a00-abe1-8316ee7aa58f-maas-api-openshift-ca\") pod \"49804909-841f-4a00-abe1-8316ee7aa58f\" (UID: \"49804909-841f-4a00-abe1-8316ee7aa58f\") "
Oct 30 19:34:14.643152 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:14.643125    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49804909-841f-4a00-abe1-8316ee7aa58f-maas-api-openshift-ca" pod "49804909-841f-4a00-abe1-8316ee7aa58f" (UID: "49804909-841f-4a00-abe1-8316ee7aa58f"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:34:14.643289 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:14.643268    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/49804909-841f-4a00-abe1-8316ee7aa58f-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:34:14.644910 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:14.644852    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49804909-841f-4a00-abe1-8316ee7aa58f-kube-api-access-wnfrg" pod "49804909-841f-4a00-abe1-8316ee7aa58f" (UID: "49804909-841f-4a00-abe1-8316ee7aa58f"). InnerVolumeSpecName "kube-api-access-wnfrg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:34:14.645353 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:14.645331    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49804909-841f-4a00-abe1-8316ee7aa58f-maas-api-xks-ca" pod "49804909-841f-4a00-abe1-8316ee7aa58f" (UID: "49804909-841f-4a00-abe1-8316ee7aa58f"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:34:14.745077 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:14.744996    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/49804909-841f-4a00-abe1-8316ee7aa58f-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:34:14.745077 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:14.745024    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wnfrg\" (UniqueName: \"kubernetes.io/projected/49804909-841f-4a00-abe1-8316ee7aa58f-kube-api-access-wnfrg\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:34:15.429558 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.429528    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-24ad9216-vcvqp"
Oct 30 19:34:15.429737 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.429558    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-24ad9216-vcvqp" event={"ID":"49804909-841f-4a00-abe1-8316ee7aa58f","Type":"ContainerDied","Data":"3df928758aac3890f85561f8a24c30060d886e8ea81c74c3e5ef2b74d8c40581"}
Oct 30 19:34:15.429737 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.429607    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3df928758aac3890f85561f8a24c30060d886e8ea81c74c3e5ef2b74d8c40581"
Oct 30 19:34:15.749517 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.749406    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"]
Oct 30 19:34:15.750066 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.750049    5907 cpu_manager.go:414] "containerMap: removing container" podUID="49804909-841f-4a00-abe1-8316ee7aa58f" containerName="revoke-keys"
Oct 30 19:34:15.750066 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.750065    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="49804909-841f-4a00-abe1-8316ee7aa58f" containerName="revoke-keys"
Oct 30 19:34:15.750225 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.750167    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="49804909-841f-4a00-abe1-8316ee7aa58f" containerName="revoke-keys"
Oct 30 19:34:15.753900 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.753842    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"
Oct 30 19:34:15.783788 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.783758    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"]
Oct 30 19:34:15.859090 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.859052    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmd7c\" (UniqueName: \"kubernetes.io/projected/1ed86722-1b31-4348-ac72-136909f507c3-kube-api-access-nmd7c\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"
Oct 30 19:34:15.859261 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.859107    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ed86722-1b31-4348-ac72-136909f507c3-istiod-ca-cert\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"
Oct 30 19:34:15.859261 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.859129    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ed86722-1b31-4348-ac72-136909f507c3-istio-data\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"
Oct 30 19:34:15.859261 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.859227    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ed86722-1b31-4348-ac72-136909f507c3-istio-token\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"
Oct 30 19:34:15.859448 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.859271    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ed86722-1b31-4348-ac72-136909f507c3-istio-envoy\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"
Oct 30 19:34:15.859448 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.859317    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ed86722-1b31-4348-ac72-136909f507c3-istio-podinfo\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"
Oct 30 19:34:15.859448 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.859423    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ed86722-1b31-4348-ac72-136909f507c3-workload-socket\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"
Oct 30 19:34:15.859593 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.859464    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ed86722-1b31-4348-ac72-136909f507c3-credential-socket\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"
Oct 30 19:34:15.859593 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.859515    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ed86722-1b31-4348-ac72-136909f507c3-workload-certs\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"
Oct 30 19:34:15.960887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.960824    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmd7c\" (UniqueName: \"kubernetes.io/projected/1ed86722-1b31-4348-ac72-136909f507c3-kube-api-access-nmd7c\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"
Oct 30 19:34:15.961041 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.960933    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ed86722-1b31-4348-ac72-136909f507c3-istiod-ca-cert\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"
Oct 30 19:34:15.961041 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.960964    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ed86722-1b31-4348-ac72-136909f507c3-istio-data\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"
Oct 30 19:34:15.961041 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.960984    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ed86722-1b31-4348-ac72-136909f507c3-istio-token\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"
Oct 30 19:34:15.961164 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.961058    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ed86722-1b31-4348-ac72-136909f507c3-istio-envoy\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"
Oct 30 19:34:15.961164 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.961119    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ed86722-1b31-4348-ac72-136909f507c3-istio-podinfo\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"
Oct 30 19:34:15.961164 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.961146    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ed86722-1b31-4348-ac72-136909f507c3-workload-socket\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"
Oct 30 19:34:15.961309 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.961174    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ed86722-1b31-4348-ac72-136909f507c3-credential-socket\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"
Oct 30 19:34:15.961309 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.961220    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ed86722-1b31-4348-ac72-136909f507c3-workload-certs\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"
Oct 30 19:34:15.961487 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.961464    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ed86722-1b31-4348-ac72-136909f507c3-istio-data\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"
Oct 30 19:34:15.961593 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.961568    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ed86722-1b31-4348-ac72-136909f507c3-credential-socket\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"
Oct 30 19:34:15.961662 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.961623    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ed86722-1b31-4348-ac72-136909f507c3-workload-socket\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"
Oct 30 19:34:15.961735 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.961715    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ed86722-1b31-4348-ac72-136909f507c3-workload-certs\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"
Oct 30 19:34:15.961781 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.961723    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ed86722-1b31-4348-ac72-136909f507c3-istiod-ca-cert\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"
Oct 30 19:34:15.963534 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.963514    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ed86722-1b31-4348-ac72-136909f507c3-istio-envoy\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"
Oct 30 19:34:15.963885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.963843    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ed86722-1b31-4348-ac72-136909f507c3-istio-podinfo\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"
Oct 30 19:34:15.983238 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.983203    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ed86722-1b31-4348-ac72-136909f507c3-istio-token\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"
Oct 30 19:34:15.983644 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:15.983626    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmd7c\" (UniqueName: \"kubernetes.io/projected/1ed86722-1b31-4348-ac72-136909f507c3-kube-api-access-nmd7c\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"
Oct 30 19:34:16.082122 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:16.082041    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"
Oct 30 19:34:16.411531 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:16.411502    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"]
Oct 30 19:34:16.412757 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:34:16.412732    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ed86722_1b31_4348_ac72_136909f507c3.slice/crio-55c08be65fb1506a8ca8ab3871af86c22e64cfa5c16dc14cd42f0353b242a8d8 WatchSource:0}: Error finding container 55c08be65fb1506a8ca8ab3871af86c22e64cfa5c16dc14cd42f0353b242a8d8: Status 404 returned error can't find the container with id 55c08be65fb1506a8ca8ab3871af86c22e64cfa5c16dc14cd42f0353b242a8d8
Oct 30 19:34:16.415807 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:16.415769    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:34:16.415899 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:16.415847    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:34:16.415943 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:16.415905    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:34:16.438828 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:16.438804    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6" event={"ID":"1ed86722-1b31-4348-ac72-136909f507c3","Type":"ContainerStarted","Data":"55c08be65fb1506a8ca8ab3871af86c22e64cfa5c16dc14cd42f0353b242a8d8"}
Oct 30 19:34:17.447851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:17.447815    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6" event={"ID":"1ed86722-1b31-4348-ac72-136909f507c3","Type":"ContainerStarted","Data":"1ffc30a47b8cf11e36e1d111813b6890868c79f41730257fa0381f107777e205"}
Oct 30 19:34:17.501317 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:34:17.501280    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:34:17.526598 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:17.526537    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6" podStartSLOduration=2.5265151279999998 podStartE2EDuration="2.526515128s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:34:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:34:17.50853267 +0000 UTC m=+769.351464642" watchObservedRunningTime="2026-10-30 19:34:17.526515128 +0000 UTC m=+769.369447067"
Oct 30 19:34:18.082524 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:18.082488    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"
Oct 30 19:34:18.083631 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:18.083607    5907 patch_prober.go:29] interesting pod/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 30 19:34:18.083749 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:18.083660    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6" podUID="1ed86722-1b31-4348-ac72-136909f507c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 30 19:34:19.097885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:19.090286    5907 patch_prober.go:29] interesting pod/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 30 19:34:19.097885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:19.090345    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6" podUID="1ed86722-1b31-4348-ac72-136909f507c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 30 19:34:20.083061 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:20.083028    5907 patch_prober.go:29] interesting pod/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 30 19:34:20.083241 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:20.083076    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6" podUID="1ed86722-1b31-4348-ac72-136909f507c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 30 19:34:21.083370 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:21.083329    5907 patch_prober.go:29] interesting pod/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 30 19:34:21.083774 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:21.083387    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6" podUID="1ed86722-1b31-4348-ac72-136909f507c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 30 19:34:22.082789 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:22.082747    5907 patch_prober.go:29] interesting pod/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 30 19:34:22.083002 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:22.082826    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6" podUID="1ed86722-1b31-4348-ac72-136909f507c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 30 19:34:23.359620 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:23.359589    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"
Oct 30 19:34:23.360965 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:23.360933    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"
Oct 30 19:34:23.362058 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:23.362039    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"
Oct 30 19:34:26.135952 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:26.135912    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:34:29.734008 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:29.733975    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"]
Oct 30 19:34:29.740015 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:29.739995    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"
Oct 30 19:34:29.780321 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:29.780288    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"]
Oct 30 19:34:29.856358 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:29.856316    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istiod-ca-cert\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"
Oct 30 19:34:29.856530 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:29.856380    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istio-envoy\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"
Oct 30 19:34:29.856574 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:29.856528    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-workload-certs\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"
Oct 30 19:34:29.856574 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:29.856561    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istio-token\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"
Oct 30 19:34:29.856696 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:29.856676    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-workload-socket\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"
Oct 30 19:34:29.856755 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:29.856712    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sc9tr\" (UniqueName: \"kubernetes.io/projected/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-kube-api-access-sc9tr\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"
Oct 30 19:34:29.856811 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:29.856769    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istio-data\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"
Oct 30 19:34:29.856919 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:29.856905    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istio-podinfo\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"
Oct 30 19:34:29.856999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:29.856951    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-credential-socket\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"
Oct 30 19:34:29.958531 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:29.958498    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-workload-socket\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"
Oct 30 19:34:29.958531 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:29.958531    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sc9tr\" (UniqueName: \"kubernetes.io/projected/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-kube-api-access-sc9tr\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"
Oct 30 19:34:29.958776 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:29.958551    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istio-data\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"
Oct 30 19:34:29.958776 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:29.958605    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istio-podinfo\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"
Oct 30 19:34:29.958776 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:29.958636    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-credential-socket\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"
Oct 30 19:34:29.958776 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:29.958688    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istiod-ca-cert\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"
Oct 30 19:34:29.958776 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:29.958726    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istio-envoy\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"
Oct 30 19:34:29.959089 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:29.958810    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-workload-certs\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"
Oct 30 19:34:29.959089 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:29.958836    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istio-token\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"
Oct 30 19:34:29.959089 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:29.959015    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-workload-socket\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"
Oct 30 19:34:29.959236 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:29.959142    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istio-data\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"
Oct 30 19:34:29.959236 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:29.959161    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-credential-socket\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"
Oct 30 19:34:29.959315 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:29.959290    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-workload-certs\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"
Oct 30 19:34:29.959459 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:29.959440    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istiod-ca-cert\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"
Oct 30 19:34:29.961657 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:29.961637    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istio-envoy\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"
Oct 30 19:34:29.961932 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:29.961912    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istio-podinfo\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"
Oct 30 19:34:29.979104 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:29.979079    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istio-token\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"
Oct 30 19:34:29.979262 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:29.979245    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sc9tr\" (UniqueName: \"kubernetes.io/projected/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-kube-api-access-sc9tr\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"
Oct 30 19:34:30.055822 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.055737    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"
Oct 30 19:34:30.288889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.288810    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"]
Oct 30 19:34:30.294661 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.294634    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"
Oct 30 19:34:30.312953 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.312925    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"]
Oct 30 19:34:30.366398 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.366350    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46317bf7-b022-49a3-b641-66d47bcfb148-istio-podinfo\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"
Oct 30 19:34:30.366575 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.366497    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46317bf7-b022-49a3-b641-66d47bcfb148-workload-certs\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"
Oct 30 19:34:30.366575 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.366541    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46317bf7-b022-49a3-b641-66d47bcfb148-istio-envoy\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"
Oct 30 19:34:30.366575 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.366564    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46317bf7-b022-49a3-b641-66d47bcfb148-workload-socket\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"
Oct 30 19:34:30.366727 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.366705    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46317bf7-b022-49a3-b641-66d47bcfb148-credential-socket\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"
Oct 30 19:34:30.366772 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.366740    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46317bf7-b022-49a3-b641-66d47bcfb148-istio-token\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"
Oct 30 19:34:30.366772 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.366756    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46317bf7-b022-49a3-b641-66d47bcfb148-istiod-ca-cert\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"
Oct 30 19:34:30.366934 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.366917    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46317bf7-b022-49a3-b641-66d47bcfb148-istio-data\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"
Oct 30 19:34:30.367015 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.366955    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7sfj\" (UniqueName: \"kubernetes.io/projected/46317bf7-b022-49a3-b641-66d47bcfb148-kube-api-access-j7sfj\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"
Oct 30 19:34:30.382597 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.382572    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"]
Oct 30 19:34:30.382994 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:34:30.382969    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6947d3d_3c9a_40d6_a9ae_0604ec1c7125.slice/crio-a45092f34486be78454dcd7f3a077648995ac2b334aa7e08ca71619b31b8129a WatchSource:0}: Error finding container a45092f34486be78454dcd7f3a077648995ac2b334aa7e08ca71619b31b8129a: Status 404 returned error can't find the container with id a45092f34486be78454dcd7f3a077648995ac2b334aa7e08ca71619b31b8129a
Oct 30 19:34:30.385331 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.385301    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:34:30.385396 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.385365    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:34:30.385437 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.385412    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:34:30.468417 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.468386    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46317bf7-b022-49a3-b641-66d47bcfb148-istio-data\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"
Oct 30 19:34:30.468417 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.468422    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7sfj\" (UniqueName: \"kubernetes.io/projected/46317bf7-b022-49a3-b641-66d47bcfb148-kube-api-access-j7sfj\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"
Oct 30 19:34:30.468692 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.468473    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46317bf7-b022-49a3-b641-66d47bcfb148-istio-podinfo\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"
Oct 30 19:34:30.468692 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.468520    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46317bf7-b022-49a3-b641-66d47bcfb148-workload-certs\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"
Oct 30 19:34:30.468692 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.468547    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46317bf7-b022-49a3-b641-66d47bcfb148-istio-envoy\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"
Oct 30 19:34:30.468692 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.468568    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46317bf7-b022-49a3-b641-66d47bcfb148-workload-socket\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"
Oct 30 19:34:30.468692 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.468616    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46317bf7-b022-49a3-b641-66d47bcfb148-credential-socket\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"
Oct 30 19:34:30.468692 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.468633    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46317bf7-b022-49a3-b641-66d47bcfb148-istio-token\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"
Oct 30 19:34:30.468692 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.468656    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46317bf7-b022-49a3-b641-66d47bcfb148-istiod-ca-cert\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"
Oct 30 19:34:30.469069 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.468929    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46317bf7-b022-49a3-b641-66d47bcfb148-istio-data\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"
Oct 30 19:34:30.469069 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.468968    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46317bf7-b022-49a3-b641-66d47bcfb148-workload-certs\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"
Oct 30 19:34:30.469069 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.469046    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46317bf7-b022-49a3-b641-66d47bcfb148-credential-socket\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"
Oct 30 19:34:30.469236 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.469214    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46317bf7-b022-49a3-b641-66d47bcfb148-workload-socket\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"
Oct 30 19:34:30.469363 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.469347    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46317bf7-b022-49a3-b641-66d47bcfb148-istiod-ca-cert\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"
Oct 30 19:34:30.471184 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.471164    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46317bf7-b022-49a3-b641-66d47bcfb148-istio-envoy\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"
Oct 30 19:34:30.471430 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.471410    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46317bf7-b022-49a3-b641-66d47bcfb148-istio-podinfo\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"
Oct 30 19:34:30.489359 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.489334    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46317bf7-b022-49a3-b641-66d47bcfb148-istio-token\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"
Oct 30 19:34:30.489516 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.489363    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7sfj\" (UniqueName: \"kubernetes.io/projected/46317bf7-b022-49a3-b641-66d47bcfb148-kube-api-access-j7sfj\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"
Oct 30 19:34:30.588953 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.588835    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq" event={"ID":"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125","Type":"ContainerStarted","Data":"e7c613909fa4ea2bd315211f18eb37a63655bee217949710516c09dd1ff9a3fa"}
Oct 30 19:34:30.588953 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.588920    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq" event={"ID":"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125","Type":"ContainerStarted","Data":"a45092f34486be78454dcd7f3a077648995ac2b334aa7e08ca71619b31b8129a"}
Oct 30 19:34:30.612699 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.612671    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"
Oct 30 19:34:30.635979 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.635933    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq" podStartSLOduration=1.635919129 podStartE2EDuration="1.635919129s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:34:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:34:30.63541032 +0000 UTC m=+782.478342261" watchObservedRunningTime="2026-10-30 19:34:30.635919129 +0000 UTC m=+782.478851068"
Oct 30 19:34:30.945265 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.945225    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"]
Oct 30 19:34:30.945620 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:34:30.945495    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46317bf7_b022_49a3_b641_66d47bcfb148.slice/crio-e57aa66b97cbbbaae5f6654fa4131dafd26a858ce79e4d1cd7183a03032744ed WatchSource:0}: Error finding container e57aa66b97cbbbaae5f6654fa4131dafd26a858ce79e4d1cd7183a03032744ed: Status 404 returned error can't find the container with id e57aa66b97cbbbaae5f6654fa4131dafd26a858ce79e4d1cd7183a03032744ed
Oct 30 19:34:30.948357 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.948312    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:34:30.948465 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.948412    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:34:30.948465 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:30.948453    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:34:31.056379 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:31.056346    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"
Oct 30 19:34:31.057928 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:31.057903    5907 patch_prober.go:29] interesting pod/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Oct 30 19:34:31.058047 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:31.057954    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq" podUID="b6947d3d-3c9a-40d6-a9ae-0604ec1c7125" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Oct 30 19:34:31.596928 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:31.596888    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt" event={"ID":"46317bf7-b022-49a3-b641-66d47bcfb148","Type":"ContainerStarted","Data":"481e1b2abb4671b891b984deeb682056d2be9d662a15fd8161624d1dae7242ec"}
Oct 30 19:34:31.597120 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:31.596949    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt" event={"ID":"46317bf7-b022-49a3-b641-66d47bcfb148","Type":"ContainerStarted","Data":"e57aa66b97cbbbaae5f6654fa4131dafd26a858ce79e4d1cd7183a03032744ed"}
Oct 30 19:34:31.621916 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:31.612810    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"
Oct 30 19:34:31.662253 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:31.662188    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt" podStartSLOduration=1.6621675599999999 podStartE2EDuration="1.66216756s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:34:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:34:31.658589536 +0000 UTC m=+783.501521476" watchObservedRunningTime="2026-10-30 19:34:31.66216756 +0000 UTC m=+783.505099499"
Oct 30 19:34:32.056390 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:32.056352    5907 patch_prober.go:29] interesting pod/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Oct 30 19:34:32.056979 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:32.056945    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq" podUID="b6947d3d-3c9a-40d6-a9ae-0604ec1c7125" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Oct 30 19:34:32.653886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:32.649460    5907 patch_prober.go:29] interesting pod/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 30 19:34:32.653886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:32.649530    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt" podUID="46317bf7-b022-49a3-b641-66d47bcfb148" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 30 19:34:33.056943 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:33.056285    5907 patch_prober.go:29] interesting pod/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Oct 30 19:34:33.056943 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:33.056357    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq" podUID="b6947d3d-3c9a-40d6-a9ae-0604ec1c7125" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Oct 30 19:34:33.628907 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:33.618958    5907 patch_prober.go:29] interesting pod/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 30 19:34:33.628907 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:33.619020    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt" podUID="46317bf7-b022-49a3-b641-66d47bcfb148" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 30 19:34:34.060943 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:34.057231    5907 patch_prober.go:29] interesting pod/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Oct 30 19:34:34.060943 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:34.057300    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq" podUID="b6947d3d-3c9a-40d6-a9ae-0604ec1c7125" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Oct 30 19:34:34.627268 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:34.627234    5907 patch_prober.go:29] interesting pod/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 30 19:34:34.627568 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:34.627541    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt" podUID="46317bf7-b022-49a3-b641-66d47bcfb148" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 30 19:34:35.057348 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:35.057241    5907 patch_prober.go:29] interesting pod/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Oct 30 19:34:35.057554 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:35.057344    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq" podUID="b6947d3d-3c9a-40d6-a9ae-0604ec1c7125" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Oct 30 19:34:35.618898 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:35.618200    5907 patch_prober.go:29] interesting pod/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 30 19:34:35.618898 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:35.618263    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt" podUID="46317bf7-b022-49a3-b641-66d47bcfb148" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 30 19:34:36.057522 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:36.057406    5907 patch_prober.go:29] interesting pod/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Oct 30 19:34:36.057840 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:36.057793    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq" podUID="b6947d3d-3c9a-40d6-a9ae-0604ec1c7125" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Oct 30 19:34:36.631888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:36.631232    5907 patch_prober.go:29] interesting pod/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 30 19:34:36.631888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:36.631309    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt" podUID="46317bf7-b022-49a3-b641-66d47bcfb148" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 30 19:34:37.056378 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:37.056284    5907 patch_prober.go:29] interesting pod/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Oct 30 19:34:37.056671 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:37.056640    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq" podUID="b6947d3d-3c9a-40d6-a9ae-0604ec1c7125" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Oct 30 19:34:37.620554 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:37.619990    5907 patch_prober.go:29] interesting pod/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 30 19:34:37.620554 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:37.620053    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt" podUID="46317bf7-b022-49a3-b641-66d47bcfb148" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 30 19:34:38.060482 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:38.060397    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"
Oct 30 19:34:38.060994 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:38.060759    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"
Oct 30 19:34:38.061374 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:38.061356    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"
Oct 30 19:34:38.617667 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:38.617628    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"
Oct 30 19:34:38.618127 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:38.618112    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"
Oct 30 19:34:38.618891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:38.618872    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"
Oct 30 19:34:47.500052 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:34:47.499997    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:34:48.487439 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:48.487403    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"]
Oct 30 19:34:48.493548 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:48.493523    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:34:48.514076 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:48.514036    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"]
Oct 30 19:34:48.521196 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:48.521166    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-f913c6\"/\"kube-root-ca.crt\""
Oct 30 19:34:48.521564 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:48.521540    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-f913c6\"/\"default-dockercfg-bcjnr\""
Oct 30 19:34:48.524244 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:48.524221    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-f913c6\"/\"distinct-2-w4-f913c6-kserve-self-signed-certs\""
Oct 30 19:34:48.524323 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:48.524284    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-f913c6\"/\"openshift-service-ca.crt\""
Oct 30 19:34:48.529933 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:48.529898    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d1837f0-48b2-446c-946c-4007adf95814-tls-certs\") pod \"distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b\" (UID: \"7d1837f0-48b2-446c-946c-4007adf95814\") " pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:34:48.530063 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:48.530042    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d1837f0-48b2-446c-946c-4007adf95814-model-cache\") pod \"distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b\" (UID: \"7d1837f0-48b2-446c-946c-4007adf95814\") " pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:34:48.530187 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:48.530172    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msfkh\" (UniqueName: \"kubernetes.io/projected/7d1837f0-48b2-446c-946c-4007adf95814-kube-api-access-msfkh\") pod \"distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b\" (UID: \"7d1837f0-48b2-446c-946c-4007adf95814\") " pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:34:48.530254 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:48.530241    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d1837f0-48b2-446c-946c-4007adf95814-home\") pod \"distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b\" (UID: \"7d1837f0-48b2-446c-946c-4007adf95814\") " pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:34:48.530303 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:48.530294    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d1837f0-48b2-446c-946c-4007adf95814-dshm\") pod \"distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b\" (UID: \"7d1837f0-48b2-446c-946c-4007adf95814\") " pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:34:48.530340 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:48.530314    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d1837f0-48b2-446c-946c-4007adf95814-tmp-dir\") pod \"distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b\" (UID: \"7d1837f0-48b2-446c-946c-4007adf95814\") " pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:34:48.631794 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:48.631761    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msfkh\" (UniqueName: \"kubernetes.io/projected/7d1837f0-48b2-446c-946c-4007adf95814-kube-api-access-msfkh\") pod \"distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b\" (UID: \"7d1837f0-48b2-446c-946c-4007adf95814\") " pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:34:48.631985 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:48.631827    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d1837f0-48b2-446c-946c-4007adf95814-home\") pod \"distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b\" (UID: \"7d1837f0-48b2-446c-946c-4007adf95814\") " pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:34:48.631985 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:48.631882    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d1837f0-48b2-446c-946c-4007adf95814-dshm\") pod \"distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b\" (UID: \"7d1837f0-48b2-446c-946c-4007adf95814\") " pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:34:48.631985 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:48.631898    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d1837f0-48b2-446c-946c-4007adf95814-tmp-dir\") pod \"distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b\" (UID: \"7d1837f0-48b2-446c-946c-4007adf95814\") " pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:34:48.631985 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:48.631950    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d1837f0-48b2-446c-946c-4007adf95814-tls-certs\") pod \"distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b\" (UID: \"7d1837f0-48b2-446c-946c-4007adf95814\") " pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:34:48.632200 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:48.632011    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d1837f0-48b2-446c-946c-4007adf95814-model-cache\") pod \"distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b\" (UID: \"7d1837f0-48b2-446c-946c-4007adf95814\") " pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:34:48.632294 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:48.632273    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d1837f0-48b2-446c-946c-4007adf95814-home\") pod \"distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b\" (UID: \"7d1837f0-48b2-446c-946c-4007adf95814\") " pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:34:48.632359 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:48.632338    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d1837f0-48b2-446c-946c-4007adf95814-tmp-dir\") pod \"distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b\" (UID: \"7d1837f0-48b2-446c-946c-4007adf95814\") " pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:34:48.632452 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:48.632427    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d1837f0-48b2-446c-946c-4007adf95814-model-cache\") pod \"distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b\" (UID: \"7d1837f0-48b2-446c-946c-4007adf95814\") " pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:34:48.634430 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:48.634406    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d1837f0-48b2-446c-946c-4007adf95814-dshm\") pod \"distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b\" (UID: \"7d1837f0-48b2-446c-946c-4007adf95814\") " pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:34:48.634693 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:48.634675    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d1837f0-48b2-446c-946c-4007adf95814-tls-certs\") pod \"distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b\" (UID: \"7d1837f0-48b2-446c-946c-4007adf95814\") " pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:34:48.652639 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:48.652610    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-msfkh\" (UniqueName: \"kubernetes.io/projected/7d1837f0-48b2-446c-946c-4007adf95814-kube-api-access-msfkh\") pod \"distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b\" (UID: \"7d1837f0-48b2-446c-946c-4007adf95814\") " pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:34:48.808283 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:48.808194    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:34:49.142221 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:34:49.142191    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7d1837f0_48b2_446c_946c_4007adf95814.slice/crio-80702fbc2cb7894232eadbb8ccda5d94d9b128a9874da474e247c0a1149566aa WatchSource:0}: Error finding container 80702fbc2cb7894232eadbb8ccda5d94d9b128a9874da474e247c0a1149566aa: Status 404 returned error can't find the container with id 80702fbc2cb7894232eadbb8ccda5d94d9b128a9874da474e247c0a1149566aa
Oct 30 19:34:49.142363 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:49.142274    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"]
Oct 30 19:34:49.859827 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:49.859770    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" event={"ID":"7d1837f0-48b2-446c-946c-4007adf95814","Type":"ContainerStarted","Data":"6a2fdccab00b205692e7d3de2df0b12e6103a08ed0d329b9e73f151942a39431"}
Oct 30 19:34:49.859827 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:49.859823    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" event={"ID":"7d1837f0-48b2-446c-946c-4007adf95814","Type":"ContainerStarted","Data":"80702fbc2cb7894232eadbb8ccda5d94d9b128a9874da474e247c0a1149566aa"}
Oct 30 19:34:49.900945 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:49.900897    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podStartSLOduration=1.900856481 podStartE2EDuration="1.900856481s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:34:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:34:49.900786144 +0000 UTC m=+801.743718083" watchObservedRunningTime="2026-10-30 19:34:49.900856481 +0000 UTC m=+801.743788420"
Oct 30 19:34:50.809436 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:50.809404    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:34:52.907920 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:52.907884    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-127a30e6-2rrkh"]
Oct 30 19:34:52.913730 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:52.913706    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-127a30e6-2rrkh"
Oct 30 19:34:52.931851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:52.931824    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-127a30e6-2rrkh"]
Oct 30 19:34:52.940008 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:52.939980    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:34:52.940152 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:52.940062    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:34:52.996628 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:52.996598    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4sjr\" (UniqueName: \"kubernetes.io/projected/8e6b567c-c205-42cb-bb15-a55eb30c9921-kube-api-access-c4sjr\") pod \"maas-api-revoke-sub-127a30e6-2rrkh\" (UID: \"8e6b567c-c205-42cb-bb15-a55eb30c9921\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-127a30e6-2rrkh"
Oct 30 19:34:52.996807 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:52.996660    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8e6b567c-c205-42cb-bb15-a55eb30c9921-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-127a30e6-2rrkh\" (UID: \"8e6b567c-c205-42cb-bb15-a55eb30c9921\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-127a30e6-2rrkh"
Oct 30 19:34:52.996807 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:52.996791    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8e6b567c-c205-42cb-bb15-a55eb30c9921-maas-api-xks-ca\") pod \"maas-api-revoke-sub-127a30e6-2rrkh\" (UID: \"8e6b567c-c205-42cb-bb15-a55eb30c9921\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-127a30e6-2rrkh"
Oct 30 19:34:53.098688 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:53.098657    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4sjr\" (UniqueName: \"kubernetes.io/projected/8e6b567c-c205-42cb-bb15-a55eb30c9921-kube-api-access-c4sjr\") pod \"maas-api-revoke-sub-127a30e6-2rrkh\" (UID: \"8e6b567c-c205-42cb-bb15-a55eb30c9921\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-127a30e6-2rrkh"
Oct 30 19:34:53.098886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:53.098746    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8e6b567c-c205-42cb-bb15-a55eb30c9921-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-127a30e6-2rrkh\" (UID: \"8e6b567c-c205-42cb-bb15-a55eb30c9921\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-127a30e6-2rrkh"
Oct 30 19:34:53.098886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:53.098807    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8e6b567c-c205-42cb-bb15-a55eb30c9921-maas-api-xks-ca\") pod \"maas-api-revoke-sub-127a30e6-2rrkh\" (UID: \"8e6b567c-c205-42cb-bb15-a55eb30c9921\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-127a30e6-2rrkh"
Oct 30 19:34:53.099392 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:53.099370    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8e6b567c-c205-42cb-bb15-a55eb30c9921-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-127a30e6-2rrkh\" (UID: \"8e6b567c-c205-42cb-bb15-a55eb30c9921\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-127a30e6-2rrkh"
Oct 30 19:34:53.101329 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:53.101310    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8e6b567c-c205-42cb-bb15-a55eb30c9921-maas-api-xks-ca\") pod \"maas-api-revoke-sub-127a30e6-2rrkh\" (UID: \"8e6b567c-c205-42cb-bb15-a55eb30c9921\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-127a30e6-2rrkh"
Oct 30 19:34:53.119873 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:53.119831    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4sjr\" (UniqueName: \"kubernetes.io/projected/8e6b567c-c205-42cb-bb15-a55eb30c9921-kube-api-access-c4sjr\") pod \"maas-api-revoke-sub-127a30e6-2rrkh\" (UID: \"8e6b567c-c205-42cb-bb15-a55eb30c9921\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-127a30e6-2rrkh"
Oct 30 19:34:53.228903 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:53.228784    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-127a30e6-2rrkh"
Oct 30 19:34:53.539834 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:53.539783    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-127a30e6-2rrkh"]
Oct 30 19:34:53.564069 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:34:53.548334    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e6b567c_c205_42cb_bb15_a55eb30c9921.slice/crio-60396102f5309ab4e82301e1f547877b68340c9dc27e518fd59c5b765d8a096e WatchSource:0}: Error finding container 60396102f5309ab4e82301e1f547877b68340c9dc27e518fd59c5b765d8a096e: Status 404 returned error can't find the container with id 60396102f5309ab4e82301e1f547877b68340c9dc27e518fd59c5b765d8a096e
Oct 30 19:34:53.890104 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:53.890069    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-127a30e6-2rrkh" event={"ID":"8e6b567c-c205-42cb-bb15-a55eb30c9921","Type":"ContainerStarted","Data":"60396102f5309ab4e82301e1f547877b68340c9dc27e518fd59c5b765d8a096e"}
Oct 30 19:34:54.899677 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:54.899643    5907 generic.go:356] "Generic (PLEG): container finished" podID="8e6b567c-c205-42cb-bb15-a55eb30c9921" containerID="75da0da1e21c466295bfa91185043b5bc76a29fb0ab995c9c9ce4940b4406cd0" exitCode=0
Oct 30 19:34:54.900113 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:54.899731    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-127a30e6-2rrkh" event={"ID":"8e6b567c-c205-42cb-bb15-a55eb30c9921","Type":"ContainerDied","Data":"75da0da1e21c466295bfa91185043b5bc76a29fb0ab995c9c9ce4940b4406cd0"}
Oct 30 19:34:56.069342 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:56.069317    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-127a30e6-2rrkh"
Oct 30 19:34:56.186314 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:56.186224    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8e6b567c-c205-42cb-bb15-a55eb30c9921-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8e6b567c-c205-42cb-bb15-a55eb30c9921-maas-api-xks-ca\") pod \"8e6b567c-c205-42cb-bb15-a55eb30c9921\" (UID: \"8e6b567c-c205-42cb-bb15-a55eb30c9921\") "
Oct 30 19:34:56.186491 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:56.186333    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8e6b567c-c205-42cb-bb15-a55eb30c9921-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8e6b567c-c205-42cb-bb15-a55eb30c9921-maas-api-openshift-ca\") pod \"8e6b567c-c205-42cb-bb15-a55eb30c9921\" (UID: \"8e6b567c-c205-42cb-bb15-a55eb30c9921\") "
Oct 30 19:34:56.186570 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:56.186503    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8e6b567c-c205-42cb-bb15-a55eb30c9921-kube-api-access-c4sjr\" (UniqueName: \"kubernetes.io/projected/8e6b567c-c205-42cb-bb15-a55eb30c9921-kube-api-access-c4sjr\") pod \"8e6b567c-c205-42cb-bb15-a55eb30c9921\" (UID: \"8e6b567c-c205-42cb-bb15-a55eb30c9921\") "
Oct 30 19:34:56.186755 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:56.186727    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e6b567c-c205-42cb-bb15-a55eb30c9921-maas-api-openshift-ca" pod "8e6b567c-c205-42cb-bb15-a55eb30c9921" (UID: "8e6b567c-c205-42cb-bb15-a55eb30c9921"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:34:56.187031 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:56.187013    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8e6b567c-c205-42cb-bb15-a55eb30c9921-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:34:56.188478 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:56.188456    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e6b567c-c205-42cb-bb15-a55eb30c9921-maas-api-xks-ca" pod "8e6b567c-c205-42cb-bb15-a55eb30c9921" (UID: "8e6b567c-c205-42cb-bb15-a55eb30c9921"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:34:56.188557 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:56.188537    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e6b567c-c205-42cb-bb15-a55eb30c9921-kube-api-access-c4sjr" pod "8e6b567c-c205-42cb-bb15-a55eb30c9921" (UID: "8e6b567c-c205-42cb-bb15-a55eb30c9921"). InnerVolumeSpecName "kube-api-access-c4sjr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:34:56.288553 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:56.288523    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4sjr\" (UniqueName: \"kubernetes.io/projected/8e6b567c-c205-42cb-bb15-a55eb30c9921-kube-api-access-c4sjr\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:34:56.288553 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:56.288549    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8e6b567c-c205-42cb-bb15-a55eb30c9921-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:34:56.929772 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:56.929739    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-127a30e6-2rrkh" event={"ID":"8e6b567c-c205-42cb-bb15-a55eb30c9921","Type":"ContainerDied","Data":"60396102f5309ab4e82301e1f547877b68340c9dc27e518fd59c5b765d8a096e"}
Oct 30 19:34:56.929772 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:56.929759    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-127a30e6-2rrkh"
Oct 30 19:34:56.930098 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:56.929796    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="60396102f5309ab4e82301e1f547877b68340c9dc27e518fd59c5b765d8a096e"
Oct 30 19:34:57.244464 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:57.244387    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:34:58.808625 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:58.808595    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:34:58.824062 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:58.824035    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:34:58.965254 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:34:58.965222    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:35:00.602961 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.602927    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"]
Oct 30 19:35:00.604508 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.604477    5907 cpu_manager.go:414] "containerMap: removing container" podUID="8e6b567c-c205-42cb-bb15-a55eb30c9921" containerName="revoke-keys"
Oct 30 19:35:00.604681 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.604667    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8e6b567c-c205-42cb-bb15-a55eb30c9921" containerName="revoke-keys"
Oct 30 19:35:00.605050 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.605032    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="8e6b567c-c205-42cb-bb15-a55eb30c9921" containerName="revoke-keys"
Oct 30 19:35:00.611648 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.611623    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"]
Oct 30 19:35:00.616045 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.616020    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"
Oct 30 19:35:00.713888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.709700    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce80464b-ca17-4aed-8228-e19549d3e1de-istio-data\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"
Oct 30 19:35:00.713888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.709776    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce80464b-ca17-4aed-8228-e19549d3e1de-credential-socket\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"
Oct 30 19:35:00.713888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.709895    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce80464b-ca17-4aed-8228-e19549d3e1de-istio-podinfo\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"
Oct 30 19:35:00.713888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.709957    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce80464b-ca17-4aed-8228-e19549d3e1de-workload-certs\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"
Oct 30 19:35:00.713888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.710122    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce80464b-ca17-4aed-8228-e19549d3e1de-istiod-ca-cert\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"
Oct 30 19:35:00.713888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.710189    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfhh4\" (UniqueName: \"kubernetes.io/projected/ce80464b-ca17-4aed-8228-e19549d3e1de-kube-api-access-hfhh4\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"
Oct 30 19:35:00.713888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.710388    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce80464b-ca17-4aed-8228-e19549d3e1de-istio-token\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"
Oct 30 19:35:00.713888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.710629    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce80464b-ca17-4aed-8228-e19549d3e1de-workload-socket\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"
Oct 30 19:35:00.713888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.710762    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce80464b-ca17-4aed-8228-e19549d3e1de-istio-envoy\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"
Oct 30 19:35:00.812490 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.812451    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce80464b-ca17-4aed-8228-e19549d3e1de-istiod-ca-cert\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"
Oct 30 19:35:00.812490 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.812491    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfhh4\" (UniqueName: \"kubernetes.io/projected/ce80464b-ca17-4aed-8228-e19549d3e1de-kube-api-access-hfhh4\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"
Oct 30 19:35:00.812718 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.812580    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce80464b-ca17-4aed-8228-e19549d3e1de-istio-token\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"
Oct 30 19:35:00.812718 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.812650    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce80464b-ca17-4aed-8228-e19549d3e1de-workload-socket\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"
Oct 30 19:35:00.812821 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.812753    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce80464b-ca17-4aed-8228-e19549d3e1de-istio-envoy\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"
Oct 30 19:35:00.813107 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.813084    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce80464b-ca17-4aed-8228-e19549d3e1de-istio-data\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"
Oct 30 19:35:00.813166 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.813150    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce80464b-ca17-4aed-8228-e19549d3e1de-workload-socket\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"
Oct 30 19:35:00.813203 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.813158    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce80464b-ca17-4aed-8228-e19549d3e1de-credential-socket\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"
Oct 30 19:35:00.813311 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.813292    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce80464b-ca17-4aed-8228-e19549d3e1de-istio-podinfo\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"
Oct 30 19:35:00.813369 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.813330    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce80464b-ca17-4aed-8228-e19549d3e1de-workload-certs\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"
Oct 30 19:35:00.813436 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.813417    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce80464b-ca17-4aed-8228-e19549d3e1de-istio-data\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"
Oct 30 19:35:00.813474 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.813450    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce80464b-ca17-4aed-8228-e19549d3e1de-credential-socket\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"
Oct 30 19:35:00.813596 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.813575    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce80464b-ca17-4aed-8228-e19549d3e1de-istiod-ca-cert\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"
Oct 30 19:35:00.813639 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.813597    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce80464b-ca17-4aed-8228-e19549d3e1de-workload-certs\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"
Oct 30 19:35:00.815459 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.815430    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce80464b-ca17-4aed-8228-e19549d3e1de-istio-envoy\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"
Oct 30 19:35:00.816118 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.816094    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce80464b-ca17-4aed-8228-e19549d3e1de-istio-podinfo\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"
Oct 30 19:35:00.835188 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.835156    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfhh4\" (UniqueName: \"kubernetes.io/projected/ce80464b-ca17-4aed-8228-e19549d3e1de-kube-api-access-hfhh4\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"
Oct 30 19:35:00.836079 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.836058    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce80464b-ca17-4aed-8228-e19549d3e1de-istio-token\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"
Oct 30 19:35:00.860405 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.860360    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"]
Oct 30 19:35:00.866502 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.866477    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"
Oct 30 19:35:00.895585 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.895539    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"]
Oct 30 19:35:00.958433 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:00.958402    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"
Oct 30 19:35:01.017414 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.017375    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a16775f-a08c-433c-8707-9eed8d73e7f9-istio-envoy\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"
Oct 30 19:35:01.017414 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.017411    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a16775f-a08c-433c-8707-9eed8d73e7f9-istio-podinfo\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"
Oct 30 19:35:01.017712 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.017686    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a16775f-a08c-433c-8707-9eed8d73e7f9-workload-certs\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"
Oct 30 19:35:01.017755 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.017734    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a16775f-a08c-433c-8707-9eed8d73e7f9-istio-data\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"
Oct 30 19:35:01.017921 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.017901    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a16775f-a08c-433c-8707-9eed8d73e7f9-workload-socket\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"
Oct 30 19:35:01.017976 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.017934    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a16775f-a08c-433c-8707-9eed8d73e7f9-istio-token\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"
Oct 30 19:35:01.017976 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.017953    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a16775f-a08c-433c-8707-9eed8d73e7f9-istiod-ca-cert\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"
Oct 30 19:35:01.018037 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.018011    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wsf5\" (UniqueName: \"kubernetes.io/projected/1a16775f-a08c-433c-8707-9eed8d73e7f9-kube-api-access-5wsf5\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"
Oct 30 19:35:01.018119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.018103    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a16775f-a08c-433c-8707-9eed8d73e7f9-credential-socket\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"
Oct 30 19:35:01.119610 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.119514    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a16775f-a08c-433c-8707-9eed8d73e7f9-workload-certs\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"
Oct 30 19:35:01.119610 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.119554    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a16775f-a08c-433c-8707-9eed8d73e7f9-istio-data\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"
Oct 30 19:35:01.119908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.119616    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a16775f-a08c-433c-8707-9eed8d73e7f9-workload-socket\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"
Oct 30 19:35:01.119908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.119634    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a16775f-a08c-433c-8707-9eed8d73e7f9-istio-token\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"
Oct 30 19:35:01.119908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.119650    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a16775f-a08c-433c-8707-9eed8d73e7f9-istiod-ca-cert\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"
Oct 30 19:35:01.119908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.119668    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wsf5\" (UniqueName: \"kubernetes.io/projected/1a16775f-a08c-433c-8707-9eed8d73e7f9-kube-api-access-5wsf5\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"
Oct 30 19:35:01.119908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.119733    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a16775f-a08c-433c-8707-9eed8d73e7f9-credential-socket\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"
Oct 30 19:35:01.119908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.119762    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a16775f-a08c-433c-8707-9eed8d73e7f9-istio-envoy\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"
Oct 30 19:35:01.119908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.119786    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a16775f-a08c-433c-8707-9eed8d73e7f9-istio-podinfo\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"
Oct 30 19:35:01.120273 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.120141    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a16775f-a08c-433c-8707-9eed8d73e7f9-workload-socket\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"
Oct 30 19:35:01.120273 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.120182    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a16775f-a08c-433c-8707-9eed8d73e7f9-istio-data\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"
Oct 30 19:35:01.120273 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.120212    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a16775f-a08c-433c-8707-9eed8d73e7f9-workload-certs\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"
Oct 30 19:35:01.120446 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.120273    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a16775f-a08c-433c-8707-9eed8d73e7f9-credential-socket\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"
Oct 30 19:35:01.120602 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.120578    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a16775f-a08c-433c-8707-9eed8d73e7f9-istiod-ca-cert\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"
Oct 30 19:35:01.122492 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.122472    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a16775f-a08c-433c-8707-9eed8d73e7f9-istio-envoy\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"
Oct 30 19:35:01.122634 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.122617    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a16775f-a08c-433c-8707-9eed8d73e7f9-istio-podinfo\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"
Oct 30 19:35:01.141081 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.141047    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a16775f-a08c-433c-8707-9eed8d73e7f9-istio-token\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"
Oct 30 19:35:01.141229 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.141139    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wsf5\" (UniqueName: \"kubernetes.io/projected/1a16775f-a08c-433c-8707-9eed8d73e7f9-kube-api-access-5wsf5\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"
Oct 30 19:35:01.198008 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.197967    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"
Oct 30 19:35:01.314034 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.313997    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"]
Oct 30 19:35:01.320468 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.318568    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:35:01.320468 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.318705    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:35:01.320468 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.318791    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:35:01.551587 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.551551    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"]
Oct 30 19:35:01.553785 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.553744    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:35:01.553948 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.553831    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:35:01.553948 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.553898    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:35:01.985731 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.983566    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv" event={"ID":"1a16775f-a08c-433c-8707-9eed8d73e7f9","Type":"ContainerStarted","Data":"f715990ad4b0265ef5f435e4c064331b3ceff62f6bbf5da249fdca18ea8e8d65"}
Oct 30 19:35:01.985731 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.983761    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv" event={"ID":"1a16775f-a08c-433c-8707-9eed8d73e7f9","Type":"ContainerStarted","Data":"7941f1931ae4b91c3dea442bad1a5800423f8facf73f60377160ebe952ec0b80"}
Oct 30 19:35:01.987953 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.987756    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm" event={"ID":"ce80464b-ca17-4aed-8228-e19549d3e1de","Type":"ContainerStarted","Data":"9949530bd3505e441f3bd1cf4907e339892d1b7f86b1e6af4a81678edaf82c5b"}
Oct 30 19:35:01.987953 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:01.987830    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm" event={"ID":"ce80464b-ca17-4aed-8228-e19549d3e1de","Type":"ContainerStarted","Data":"ab972325969c5f2dfa535f991749e40dbed83d4a1597d4a6b09ea7ab6d697ed9"}
Oct 30 19:35:02.138074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:02.137990    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm" podStartSLOduration=2.137965662 podStartE2EDuration="2.137965662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:35:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:35:02.109059019 +0000 UTC m=+813.951990957" watchObservedRunningTime="2026-10-30 19:35:02.137965662 +0000 UTC m=+813.980897602"
Oct 30 19:35:02.141326 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:02.141267    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv" podStartSLOduration=2.141246942 podStartE2EDuration="2.141246942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:35:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:35:02.048002916 +0000 UTC m=+813.890934855" watchObservedRunningTime="2026-10-30 19:35:02.141246942 +0000 UTC m=+813.984178884"
Oct 30 19:35:02.198469 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:02.198431    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"
Oct 30 19:35:02.200100 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:02.200073    5907 patch_prober.go:29] interesting pod/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 19:35:02.200231 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:02.200121    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv" podUID="1a16775f-a08c-433c-8707-9eed8d73e7f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 19:35:02.960942 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:02.960747    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"
Oct 30 19:35:02.961151 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:02.961009    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 30 19:35:02.961151 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:02.961075    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm" podUID="ce80464b-ca17-4aed-8228-e19549d3e1de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 30 19:35:03.198806 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:03.198774    5907 patch_prober.go:29] interesting pod/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 19:35:03.199223 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:03.198821    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv" podUID="1a16775f-a08c-433c-8707-9eed8d73e7f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 19:35:03.961066 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:03.961013    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 30 19:35:03.961299 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:03.961098    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm" podUID="ce80464b-ca17-4aed-8228-e19549d3e1de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 30 19:35:04.198416 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:04.198377    5907 patch_prober.go:29] interesting pod/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 19:35:04.198617 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:04.198438    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv" podUID="1a16775f-a08c-433c-8707-9eed8d73e7f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 19:35:04.960668 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:04.960624    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 30 19:35:04.961328 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:04.961285    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm" podUID="ce80464b-ca17-4aed-8228-e19549d3e1de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 30 19:35:05.199390 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:05.199338    5907 patch_prober.go:29] interesting pod/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 19:35:05.199778 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:05.199727    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv" podUID="1a16775f-a08c-433c-8707-9eed8d73e7f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 19:35:05.962319 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:05.962279    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 30 19:35:05.962977 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:05.962940    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm" podUID="ce80464b-ca17-4aed-8228-e19549d3e1de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 30 19:35:06.198647 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:06.198609    5907 patch_prober.go:29] interesting pod/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 19:35:06.198828 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:06.198661    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv" podUID="1a16775f-a08c-433c-8707-9eed8d73e7f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 19:35:06.807032 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:06.806993    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d9225589-5hss5"]
Oct 30 19:35:06.813977 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:06.811892    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d9225589-5hss5"
Oct 30 19:35:06.841918 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:06.841889    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:35:06.842275 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:06.842260    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:35:06.871395 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:06.871350    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d9225589-5hss5"]
Oct 30 19:35:06.958251 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:06.957768    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d7ee7826-fc21-4470-9ae7-1db1fb50c50d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d9225589-5hss5\" (UID: \"d7ee7826-fc21-4470-9ae7-1db1fb50c50d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d9225589-5hss5"
Oct 30 19:35:06.958251 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:06.957940    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdv9l\" (UniqueName: \"kubernetes.io/projected/d7ee7826-fc21-4470-9ae7-1db1fb50c50d-kube-api-access-xdv9l\") pod \"maas-api-revoke-sub-d9225589-5hss5\" (UID: \"d7ee7826-fc21-4470-9ae7-1db1fb50c50d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d9225589-5hss5"
Oct 30 19:35:06.958251 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:06.958009    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d7ee7826-fc21-4470-9ae7-1db1fb50c50d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d9225589-5hss5\" (UID: \"d7ee7826-fc21-4470-9ae7-1db1fb50c50d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d9225589-5hss5"
Oct 30 19:35:06.963038 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:06.963008    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 30 19:35:06.963633 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:06.963604    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm" podUID="ce80464b-ca17-4aed-8228-e19549d3e1de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 30 19:35:07.060950 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:07.060270    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d7ee7826-fc21-4470-9ae7-1db1fb50c50d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d9225589-5hss5\" (UID: \"d7ee7826-fc21-4470-9ae7-1db1fb50c50d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d9225589-5hss5"
Oct 30 19:35:07.060950 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:07.060438    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdv9l\" (UniqueName: \"kubernetes.io/projected/d7ee7826-fc21-4470-9ae7-1db1fb50c50d-kube-api-access-xdv9l\") pod \"maas-api-revoke-sub-d9225589-5hss5\" (UID: \"d7ee7826-fc21-4470-9ae7-1db1fb50c50d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d9225589-5hss5"
Oct 30 19:35:07.060950 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:07.060532    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d7ee7826-fc21-4470-9ae7-1db1fb50c50d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d9225589-5hss5\" (UID: \"d7ee7826-fc21-4470-9ae7-1db1fb50c50d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d9225589-5hss5"
Oct 30 19:35:07.061604 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:07.061537    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d7ee7826-fc21-4470-9ae7-1db1fb50c50d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d9225589-5hss5\" (UID: \"d7ee7826-fc21-4470-9ae7-1db1fb50c50d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d9225589-5hss5"
Oct 30 19:35:07.063499 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:07.063474    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d7ee7826-fc21-4470-9ae7-1db1fb50c50d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d9225589-5hss5\" (UID: \"d7ee7826-fc21-4470-9ae7-1db1fb50c50d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d9225589-5hss5"
Oct 30 19:35:07.083367 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:07.083341    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdv9l\" (UniqueName: \"kubernetes.io/projected/d7ee7826-fc21-4470-9ae7-1db1fb50c50d-kube-api-access-xdv9l\") pod \"maas-api-revoke-sub-d9225589-5hss5\" (UID: \"d7ee7826-fc21-4470-9ae7-1db1fb50c50d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d9225589-5hss5"
Oct 30 19:35:07.172901 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:07.172846    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d9225589-5hss5"
Oct 30 19:35:07.202386 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:07.202351    5907 patch_prober.go:29] interesting pod/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 19:35:07.202656 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:07.202629    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv" podUID="1a16775f-a08c-433c-8707-9eed8d73e7f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 19:35:07.636582 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:07.636529    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d9225589-5hss5"]
Oct 30 19:35:07.641940 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:35:07.641907    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7ee7826_fc21_4470_9ae7_1db1fb50c50d.slice/crio-dd9205c975cb4430e3b0f48570c4f6eebf4bbdf26050f53d80c7490a8637e83a WatchSource:0}: Error finding container dd9205c975cb4430e3b0f48570c4f6eebf4bbdf26050f53d80c7490a8637e83a: Status 404 returned error can't find the container with id dd9205c975cb4430e3b0f48570c4f6eebf4bbdf26050f53d80c7490a8637e83a
Oct 30 19:35:07.959099 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:07.958999    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 30 19:35:07.959099 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:07.959073    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm" podUID="ce80464b-ca17-4aed-8228-e19549d3e1de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 30 19:35:08.083776 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:08.083740    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d9225589-5hss5" event={"ID":"d7ee7826-fc21-4470-9ae7-1db1fb50c50d","Type":"ContainerStarted","Data":"cc0d24da48a6975020a162fb2520b9c890b27e8eb96b9547e1c6704cfe030587"}
Oct 30 19:35:08.084186 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:08.083795    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d9225589-5hss5" event={"ID":"d7ee7826-fc21-4470-9ae7-1db1fb50c50d","Type":"ContainerStarted","Data":"dd9205c975cb4430e3b0f48570c4f6eebf4bbdf26050f53d80c7490a8637e83a"}
Oct 30 19:35:08.145972 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:08.142557    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d9225589-5hss5" podStartSLOduration=2.142534138 podStartE2EDuration="2.142534138s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:35:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:35:08.124096316 +0000 UTC m=+819.967028248" watchObservedRunningTime="2026-10-30 19:35:08.142534138 +0000 UTC m=+819.985466080"
Oct 30 19:35:08.198397 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:08.198353    5907 patch_prober.go:29] interesting pod/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 19:35:08.198576 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:08.198422    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv" podUID="1a16775f-a08c-433c-8707-9eed8d73e7f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 19:35:08.963287 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:08.963254    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"
Oct 30 19:35:08.963597 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:08.963573    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"
Oct 30 19:35:08.964442 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:08.964421    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"
Oct 30 19:35:09.098094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:09.098007    5907 generic.go:356] "Generic (PLEG): container finished" podID="d7ee7826-fc21-4470-9ae7-1db1fb50c50d" containerID="cc0d24da48a6975020a162fb2520b9c890b27e8eb96b9547e1c6704cfe030587" exitCode=0
Oct 30 19:35:09.098460 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:09.098107    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d9225589-5hss5" event={"ID":"d7ee7826-fc21-4470-9ae7-1db1fb50c50d","Type":"ContainerDied","Data":"cc0d24da48a6975020a162fb2520b9c890b27e8eb96b9547e1c6704cfe030587"}
Oct 30 19:35:09.199250 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:09.199215    5907 patch_prober.go:29] interesting pod/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 19:35:09.199408 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:09.199310    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv" podUID="1a16775f-a08c-433c-8707-9eed8d73e7f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 19:35:10.203637 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:10.203601    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"
Oct 30 19:35:10.204489 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:10.204284    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"
Oct 30 19:35:10.205389 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:10.205363    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"
Oct 30 19:35:10.276175 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:10.276148    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d9225589-5hss5"
Oct 30 19:35:10.451373 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:10.448699    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d7ee7826-fc21-4470-9ae7-1db1fb50c50d-kube-api-access-xdv9l\" (UniqueName: \"kubernetes.io/projected/d7ee7826-fc21-4470-9ae7-1db1fb50c50d-kube-api-access-xdv9l\") pod \"d7ee7826-fc21-4470-9ae7-1db1fb50c50d\" (UID: \"d7ee7826-fc21-4470-9ae7-1db1fb50c50d\") "
Oct 30 19:35:10.451373 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:10.448904    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d7ee7826-fc21-4470-9ae7-1db1fb50c50d-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d7ee7826-fc21-4470-9ae7-1db1fb50c50d-maas-api-xks-ca\") pod \"d7ee7826-fc21-4470-9ae7-1db1fb50c50d\" (UID: \"d7ee7826-fc21-4470-9ae7-1db1fb50c50d\") "
Oct 30 19:35:10.451373 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:10.448974    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d7ee7826-fc21-4470-9ae7-1db1fb50c50d-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d7ee7826-fc21-4470-9ae7-1db1fb50c50d-maas-api-openshift-ca\") pod \"d7ee7826-fc21-4470-9ae7-1db1fb50c50d\" (UID: \"d7ee7826-fc21-4470-9ae7-1db1fb50c50d\") "
Oct 30 19:35:10.486963 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:10.486093    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7ee7826-fc21-4470-9ae7-1db1fb50c50d-maas-api-openshift-ca" pod "d7ee7826-fc21-4470-9ae7-1db1fb50c50d" (UID: "d7ee7826-fc21-4470-9ae7-1db1fb50c50d"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:35:10.620839 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:10.620796    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d7ee7826-fc21-4470-9ae7-1db1fb50c50d-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:35:10.643955 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:10.638559    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7ee7826-fc21-4470-9ae7-1db1fb50c50d-kube-api-access-xdv9l" pod "d7ee7826-fc21-4470-9ae7-1db1fb50c50d" (UID: "d7ee7826-fc21-4470-9ae7-1db1fb50c50d"). InnerVolumeSpecName "kube-api-access-xdv9l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:35:10.643955 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:10.641539    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7ee7826-fc21-4470-9ae7-1db1fb50c50d-maas-api-xks-ca" pod "d7ee7826-fc21-4470-9ae7-1db1fb50c50d" (UID: "d7ee7826-fc21-4470-9ae7-1db1fb50c50d"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:35:10.722066 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:10.722026    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d7ee7826-fc21-4470-9ae7-1db1fb50c50d-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:35:10.722066 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:10.722061    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdv9l\" (UniqueName: \"kubernetes.io/projected/d7ee7826-fc21-4470-9ae7-1db1fb50c50d-kube-api-access-xdv9l\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:35:11.161886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:11.158014    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d9225589-5hss5"
Oct 30 19:35:11.183920 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:11.174052    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d9225589-5hss5" event={"ID":"d7ee7826-fc21-4470-9ae7-1db1fb50c50d","Type":"ContainerDied","Data":"dd9205c975cb4430e3b0f48570c4f6eebf4bbdf26050f53d80c7490a8637e83a"}
Oct 30 19:35:11.183920 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:11.174125    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dd9205c975cb4430e3b0f48570c4f6eebf4bbdf26050f53d80c7490a8637e83a"
Oct 30 19:35:12.750889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:12.744975    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:35:12.750889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:12.745415    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:35:12.959028 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:12.958982    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": context deadline exceeded"
Oct 30 19:35:13.314027 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:13.312320    5907 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/health\": context deadline exceeded"
Oct 30 19:35:13.759895 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:13.758842    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:35:13.764674 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:13.763506    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:35:13.891920 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:13.891794    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:35:14.877964 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:14.877930    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"]
Oct 30 19:35:14.879138 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:14.879113    5907 cpu_manager.go:414] "containerMap: removing container" podUID="d7ee7826-fc21-4470-9ae7-1db1fb50c50d" containerName="revoke-keys"
Oct 30 19:35:14.879138 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:14.879135    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d7ee7826-fc21-4470-9ae7-1db1fb50c50d" containerName="revoke-keys"
Oct 30 19:35:14.879285 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:14.879223    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="d7ee7826-fc21-4470-9ae7-1db1fb50c50d" containerName="revoke-keys"
Oct 30 19:35:14.882999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:14.882980    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"
Oct 30 19:35:14.915303 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:14.915245    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"]
Oct 30 19:35:14.924493 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:14.924308    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"]
Oct 30 19:35:14.924777 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:14.924520    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"
Oct 30 19:35:14.931039 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:14.931012    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"]
Oct 30 19:35:14.940510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:14.940156    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"
Oct 30 19:35:14.955382 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:14.955354    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"]
Oct 30 19:35:14.962317 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:14.962292    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"]
Oct 30 19:35:15.003781 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.003745    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27e1950d-b453-4f32-afd2-95ab9027a2e5-workload-socket\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"
Oct 30 19:35:15.004100 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.004069    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27e1950d-b453-4f32-afd2-95ab9027a2e5-credential-socket\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"
Oct 30 19:35:15.004284 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.004263    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27e1950d-b453-4f32-afd2-95ab9027a2e5-workload-certs\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"
Oct 30 19:35:15.004372 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.004309    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcllf\" (UniqueName: \"kubernetes.io/projected/27e1950d-b453-4f32-afd2-95ab9027a2e5-kube-api-access-gcllf\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"
Oct 30 19:35:15.004458 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.004426    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27e1950d-b453-4f32-afd2-95ab9027a2e5-istiod-ca-cert\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"
Oct 30 19:35:15.004563 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.004506    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27e1950d-b453-4f32-afd2-95ab9027a2e5-istio-token\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"
Oct 30 19:35:15.004563 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.004540    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27e1950d-b453-4f32-afd2-95ab9027a2e5-istio-podinfo\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"
Oct 30 19:35:15.004682 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.004586    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27e1950d-b453-4f32-afd2-95ab9027a2e5-istio-data\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"
Oct 30 19:35:15.004682 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.004604    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27e1950d-b453-4f32-afd2-95ab9027a2e5-istio-envoy\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"
Oct 30 19:35:15.064559 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.064523    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"]
Oct 30 19:35:15.069662 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.069633    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"
Oct 30 19:35:15.097002 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.096968    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"]
Oct 30 19:35:15.106444 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.106395    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71285665-ae23-4783-a24f-535dccfb68c7-credential-socket\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"
Oct 30 19:35:15.106626 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.106453    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-credential-socket\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"
Oct 30 19:35:15.106684 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.106624    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27e1950d-b453-4f32-afd2-95ab9027a2e5-workload-certs\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"
Oct 30 19:35:15.106740 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.106669    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-istio-token\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"
Oct 30 19:35:15.106740 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.106725    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcllf\" (UniqueName: \"kubernetes.io/projected/27e1950d-b453-4f32-afd2-95ab9027a2e5-kube-api-access-gcllf\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"
Oct 30 19:35:15.106809 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.106767    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27e1950d-b453-4f32-afd2-95ab9027a2e5-istiod-ca-cert\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"
Oct 30 19:35:15.106843 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.106803    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-istio-podinfo\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"
Oct 30 19:35:15.106909 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.106895    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71285665-ae23-4783-a24f-535dccfb68c7-istio-podinfo\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"
Oct 30 19:35:15.106961 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.106941    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27e1950d-b453-4f32-afd2-95ab9027a2e5-istio-token\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"
Oct 30 19:35:15.107009 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.106978    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27e1950d-b453-4f32-afd2-95ab9027a2e5-istio-podinfo\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"
Oct 30 19:35:15.107041 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.107018    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-workload-socket\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"
Oct 30 19:35:15.107081 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.107064    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71285665-ae23-4783-a24f-535dccfb68c7-istio-envoy\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"
Oct 30 19:35:15.107128 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.107113    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27e1950d-b453-4f32-afd2-95ab9027a2e5-istio-data\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"
Oct 30 19:35:15.107160 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.107150    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27e1950d-b453-4f32-afd2-95ab9027a2e5-istio-envoy\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"
Oct 30 19:35:15.107201 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.107180    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnxsg\" (UniqueName: \"kubernetes.io/projected/71285665-ae23-4783-a24f-535dccfb68c7-kube-api-access-xnxsg\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"
Oct 30 19:35:15.107239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.107219    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71285665-ae23-4783-a24f-535dccfb68c7-istio-data\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"
Oct 30 19:35:15.107282 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.107272    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-istiod-ca-cert\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"
Oct 30 19:35:15.107371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.107356    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71285665-ae23-4783-a24f-535dccfb68c7-workload-socket\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"
Oct 30 19:35:15.107406 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.107394    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27e1950d-b453-4f32-afd2-95ab9027a2e5-workload-socket\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"
Oct 30 19:35:15.107439 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.107421    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27e1950d-b453-4f32-afd2-95ab9027a2e5-credential-socket\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"
Oct 30 19:35:15.107470 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.107453    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71285665-ae23-4783-a24f-535dccfb68c7-workload-certs\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"
Oct 30 19:35:15.107504 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.107478    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71285665-ae23-4783-a24f-535dccfb68c7-istiod-ca-cert\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"
Oct 30 19:35:15.107537 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.107503    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-workload-certs\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"
Oct 30 19:35:15.107572 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.107532    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-istio-envoy\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"
Oct 30 19:35:15.107608 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.107580    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncwc7\" (UniqueName: \"kubernetes.io/projected/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-kube-api-access-ncwc7\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"
Oct 30 19:35:15.107668 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.107653    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71285665-ae23-4783-a24f-535dccfb68c7-istio-token\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"
Oct 30 19:35:15.107724 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.107709    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-istio-data\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"
Oct 30 19:35:15.108362 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.108339    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27e1950d-b453-4f32-afd2-95ab9027a2e5-workload-certs\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"
Oct 30 19:35:15.109388 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.109364    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27e1950d-b453-4f32-afd2-95ab9027a2e5-istiod-ca-cert\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"
Oct 30 19:35:15.110426 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.110292    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27e1950d-b453-4f32-afd2-95ab9027a2e5-istio-data\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"
Oct 30 19:35:15.110699 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.110638    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27e1950d-b453-4f32-afd2-95ab9027a2e5-workload-socket\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"
Oct 30 19:35:15.110699 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.110664    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27e1950d-b453-4f32-afd2-95ab9027a2e5-credential-socket\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"
Oct 30 19:35:15.112330 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.112310    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27e1950d-b453-4f32-afd2-95ab9027a2e5-istio-envoy\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"
Oct 30 19:35:15.112584 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.112561    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27e1950d-b453-4f32-afd2-95ab9027a2e5-istio-podinfo\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"
Oct 30 19:35:15.138630 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.138559    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27e1950d-b453-4f32-afd2-95ab9027a2e5-istio-token\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"
Oct 30 19:35:15.138903 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.138881    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcllf\" (UniqueName: \"kubernetes.io/projected/27e1950d-b453-4f32-afd2-95ab9027a2e5-kube-api-access-gcllf\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"
Oct 30 19:35:15.237803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.213071    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/210e8e86-7046-4152-9a79-8f3c0fcdc900-istio-token\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"
Oct 30 19:35:15.238198 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.238162    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/210e8e86-7046-4152-9a79-8f3c0fcdc900-credential-socket\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"
Oct 30 19:35:15.238374 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.238359    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-istiod-ca-cert\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"
Oct 30 19:35:15.238490 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.238476    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71285665-ae23-4783-a24f-535dccfb68c7-istiod-ca-cert\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"
Oct 30 19:35:15.238600 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.238585    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-workload-certs\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"
Oct 30 19:35:15.238760 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.238736    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-istio-envoy\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"
Oct 30 19:35:15.246086 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.246054    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/210e8e86-7046-4152-9a79-8f3c0fcdc900-workload-socket\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"
Oct 30 19:35:15.246294 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.214160    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"
Oct 30 19:35:15.246458 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.245679    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-workload-certs\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"
Oct 30 19:35:15.246551 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.245700    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71285665-ae23-4783-a24f-535dccfb68c7-istiod-ca-cert\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"
Oct 30 19:35:15.246551 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.245333    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-istiod-ca-cert\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"
Oct 30 19:35:15.246551 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.246347    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/210e8e86-7046-4152-9a79-8f3c0fcdc900-istio-data\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"
Oct 30 19:35:15.246690 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.246612    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/210e8e86-7046-4152-9a79-8f3c0fcdc900-workload-certs\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"
Oct 30 19:35:15.246740 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.246705    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71285665-ae23-4783-a24f-535dccfb68c7-credential-socket\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"
Oct 30 19:35:15.255194 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.254294    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71285665-ae23-4783-a24f-535dccfb68c7-credential-socket\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"
Oct 30 19:35:15.280223 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.264631    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-istio-envoy\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"
Oct 30 19:35:15.280223 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.277203    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71285665-ae23-4783-a24f-535dccfb68c7-istio-podinfo\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"
Oct 30 19:35:15.280223 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.277280    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-workload-socket\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"
Oct 30 19:35:15.280223 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.277765    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-workload-socket\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"
Oct 30 19:35:15.280223 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.278358    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71285665-ae23-4783-a24f-535dccfb68c7-istio-envoy\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"
Oct 30 19:35:15.280223 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.278389    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71285665-ae23-4783-a24f-535dccfb68c7-workload-socket\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"
Oct 30 19:35:15.280223 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.278431    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnxsg\" (UniqueName: \"kubernetes.io/projected/71285665-ae23-4783-a24f-535dccfb68c7-kube-api-access-xnxsg\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"
Oct 30 19:35:15.280223 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.278469    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71285665-ae23-4783-a24f-535dccfb68c7-istio-data\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"
Oct 30 19:35:15.280223 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.278521    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/210e8e86-7046-4152-9a79-8f3c0fcdc900-istio-envoy\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"
Oct 30 19:35:15.280223 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.278597    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71285665-ae23-4783-a24f-535dccfb68c7-workload-certs\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"
Oct 30 19:35:15.280223 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.278630    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/210e8e86-7046-4152-9a79-8f3c0fcdc900-istiod-ca-cert\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"
Oct 30 19:35:15.280223 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.278674    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncwc7\" (UniqueName: \"kubernetes.io/projected/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-kube-api-access-ncwc7\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"
Oct 30 19:35:15.280223 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.278715    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71285665-ae23-4783-a24f-535dccfb68c7-istio-token\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"
Oct 30 19:35:15.280223 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.278762    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-credential-socket\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"
Oct 30 19:35:15.280223 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.278793    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-istio-data\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"
Oct 30 19:35:15.280223 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.278889    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-istio-token\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"
Oct 30 19:35:15.280223 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.278924    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrcnp\" (UniqueName: \"kubernetes.io/projected/210e8e86-7046-4152-9a79-8f3c0fcdc900-kube-api-access-jrcnp\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"
Oct 30 19:35:15.280223 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.278999    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-istio-podinfo\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"
Oct 30 19:35:15.280223 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.279072    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/210e8e86-7046-4152-9a79-8f3c0fcdc900-istio-podinfo\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"
Oct 30 19:35:15.281517 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.281480    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-credential-socket\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"
Oct 30 19:35:15.281943 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.281735    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-istio-data\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"
Oct 30 19:35:15.290249 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.283224    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71285665-ae23-4783-a24f-535dccfb68c7-istio-data\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"
Oct 30 19:35:15.290249 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.283311    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71285665-ae23-4783-a24f-535dccfb68c7-workload-certs\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"
Oct 30 19:35:15.290249 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.283750    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71285665-ae23-4783-a24f-535dccfb68c7-workload-socket\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"
Oct 30 19:35:15.290249 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.289615    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71285665-ae23-4783-a24f-535dccfb68c7-istio-envoy\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"
Oct 30 19:35:15.290607 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.290301    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71285665-ae23-4783-a24f-535dccfb68c7-istio-podinfo\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"
Oct 30 19:35:15.292788 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.292636    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-istio-podinfo\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"
Oct 30 19:35:15.306171 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.306129    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-istio-token\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"
Oct 30 19:35:15.309332 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.309296    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnxsg\" (UniqueName: \"kubernetes.io/projected/71285665-ae23-4783-a24f-535dccfb68c7-kube-api-access-xnxsg\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"
Oct 30 19:35:15.309499 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.309404    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71285665-ae23-4783-a24f-535dccfb68c7-istio-token\") pod \"e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") " pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"
Oct 30 19:35:15.309587 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.309517    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncwc7\" (UniqueName: \"kubernetes.io/projected/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-kube-api-access-ncwc7\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"
Oct 30 19:35:15.380996 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.380957    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrcnp\" (UniqueName: \"kubernetes.io/projected/210e8e86-7046-4152-9a79-8f3c0fcdc900-kube-api-access-jrcnp\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"
Oct 30 19:35:15.381136 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.381035    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/210e8e86-7046-4152-9a79-8f3c0fcdc900-istio-podinfo\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"
Oct 30 19:35:15.381136 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.381083    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/210e8e86-7046-4152-9a79-8f3c0fcdc900-istio-token\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"
Oct 30 19:35:15.381225 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.381138    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/210e8e86-7046-4152-9a79-8f3c0fcdc900-credential-socket\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"
Oct 30 19:35:15.381225 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.381203    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/210e8e86-7046-4152-9a79-8f3c0fcdc900-workload-socket\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"
Oct 30 19:35:15.381302 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.381228    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/210e8e86-7046-4152-9a79-8f3c0fcdc900-istio-data\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"
Oct 30 19:35:15.381302 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.381285    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/210e8e86-7046-4152-9a79-8f3c0fcdc900-workload-certs\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"
Oct 30 19:35:15.382395 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.381470    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/210e8e86-7046-4152-9a79-8f3c0fcdc900-istio-envoy\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"
Oct 30 19:35:15.382395 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.381547    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/210e8e86-7046-4152-9a79-8f3c0fcdc900-istiod-ca-cert\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"
Oct 30 19:35:15.382395 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.381724    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/210e8e86-7046-4152-9a79-8f3c0fcdc900-istio-data\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"
Oct 30 19:35:15.382395 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.381843    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/210e8e86-7046-4152-9a79-8f3c0fcdc900-credential-socket\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"
Oct 30 19:35:15.382395 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.382021    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/210e8e86-7046-4152-9a79-8f3c0fcdc900-workload-socket\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"
Oct 30 19:35:15.382395 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.382318    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/210e8e86-7046-4152-9a79-8f3c0fcdc900-istiod-ca-cert\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"
Oct 30 19:35:15.382781 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.382472    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/210e8e86-7046-4152-9a79-8f3c0fcdc900-workload-certs\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"
Oct 30 19:35:15.384095 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.384075    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/210e8e86-7046-4152-9a79-8f3c0fcdc900-istio-podinfo\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"
Oct 30 19:35:15.385438 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.385412    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/210e8e86-7046-4152-9a79-8f3c0fcdc900-istio-envoy\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"
Oct 30 19:35:15.402523 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.402455    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrcnp\" (UniqueName: \"kubernetes.io/projected/210e8e86-7046-4152-9a79-8f3c0fcdc900-kube-api-access-jrcnp\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"
Oct 30 19:35:15.404212 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.404191    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/210e8e86-7046-4152-9a79-8f3c0fcdc900-istio-token\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"
Oct 30 19:35:15.569982 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.569953    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"
Oct 30 19:35:15.573996 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.573974    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"
Oct 30 19:35:15.627622 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.627589    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"]
Oct 30 19:35:15.631240 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:35:15.631209    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27e1950d_b453_4f32_afd2_95ab9027a2e5.slice/crio-4d954e078d0018c60b7ecc546193ae3c180ab53c8dcc0fb05885ac0675234a3e WatchSource:0}: Error finding container 4d954e078d0018c60b7ecc546193ae3c180ab53c8dcc0fb05885ac0675234a3e: Status 404 returned error can't find the container with id 4d954e078d0018c60b7ecc546193ae3c180ab53c8dcc0fb05885ac0675234a3e
Oct 30 19:35:15.639759 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.639721    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:35:15.639894 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.639805    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:35:15.639894 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.639837    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:35:15.698312 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.696923    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"
Oct 30 19:35:15.919924 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:35:15.919895    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod452bcb07_8aa6_4e70_8f3a_af8bfaf756a4.slice/crio-49d2b426df4d957089e4502d27e2064ecfb1422ef9e00b746ec43facee77db06 WatchSource:0}: Error finding container 49d2b426df4d957089e4502d27e2064ecfb1422ef9e00b746ec43facee77db06: Status 404 returned error can't find the container with id 49d2b426df4d957089e4502d27e2064ecfb1422ef9e00b746ec43facee77db06
Oct 30 19:35:15.920366 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.920294    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"]
Oct 30 19:35:15.924575 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.924532    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:35:15.924667 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.924605    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:35:15.924667 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.924636    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:35:15.926602 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.926575    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"]
Oct 30 19:35:15.938298 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:35:15.938268    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71285665_ae23_4783_a24f_535dccfb68c7.slice/crio-426dfbe93bb9bf3f6116209daa2db778603675f13cb0ffa12aefadfa22f6669a WatchSource:0}: Error finding container 426dfbe93bb9bf3f6116209daa2db778603675f13cb0ffa12aefadfa22f6669a: Status 404 returned error can't find the container with id 426dfbe93bb9bf3f6116209daa2db778603675f13cb0ffa12aefadfa22f6669a
Oct 30 19:35:15.941593 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.941549    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:35:15.941700 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.941639    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:35:15.941700 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:15.941679    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:35:16.039916 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:16.039882    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"]
Oct 30 19:35:16.042615 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:16.042576    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:35:16.042737 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:16.042648    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:35:16.042737 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:16.042683    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:35:16.331242 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:16.331147    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np" event={"ID":"71285665-ae23-4783-a24f-535dccfb68c7","Type":"ContainerStarted","Data":"7e0b09bd4ae355d8392bff6d6599fdfe47e0360a686924bf9de3595f210fff14"}
Oct 30 19:35:16.331506 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:16.331483    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np" event={"ID":"71285665-ae23-4783-a24f-535dccfb68c7","Type":"ContainerStarted","Data":"426dfbe93bb9bf3f6116209daa2db778603675f13cb0ffa12aefadfa22f6669a"}
Oct 30 19:35:16.334506 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:16.334471    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c" event={"ID":"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4","Type":"ContainerStarted","Data":"3808fa08aab4443d03b99268d763f7d40e24c80c9eb5d899e96a30e6870e932c"}
Oct 30 19:35:16.334623 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:16.334529    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c" event={"ID":"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4","Type":"ContainerStarted","Data":"49d2b426df4d957089e4502d27e2064ecfb1422ef9e00b746ec43facee77db06"}
Oct 30 19:35:16.337527 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:16.337497    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4" event={"ID":"27e1950d-b453-4f32-afd2-95ab9027a2e5","Type":"ContainerStarted","Data":"6ef915f651c8f45e0b04b63b9bb1e6c7d25192c102ac8f92036f1cea6a57e038"}
Oct 30 19:35:16.338598 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:16.338574    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4" event={"ID":"27e1950d-b453-4f32-afd2-95ab9027a2e5","Type":"ContainerStarted","Data":"4d954e078d0018c60b7ecc546193ae3c180ab53c8dcc0fb05885ac0675234a3e"}
Oct 30 19:35:16.341080 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:16.341060    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl" event={"ID":"210e8e86-7046-4152-9a79-8f3c0fcdc900","Type":"ContainerStarted","Data":"124ee5b9d32da880da7d58796c17736fda5164ae6c1efc6bb1a6bb003f1a4144"}
Oct 30 19:35:16.341225 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:16.341208    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl" event={"ID":"210e8e86-7046-4152-9a79-8f3c0fcdc900","Type":"ContainerStarted","Data":"bdec10a710d320d834c5807711c03b7eaafc916e6d6cff5afff4b027651e513d"}
Oct 30 19:35:16.389911 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:16.389682    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np" podStartSLOduration=2.38966206 podStartE2EDuration="2.38966206s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:35:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:35:16.387423693 +0000 UTC m=+828.230355644" watchObservedRunningTime="2026-10-30 19:35:16.38966206 +0000 UTC m=+828.232594014"
Oct 30 19:35:16.453886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:16.453292    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c" podStartSLOduration=2.453270899 podStartE2EDuration="2.453270899s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:35:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:35:16.432126004 +0000 UTC m=+828.275057944" watchObservedRunningTime="2026-10-30 19:35:16.453270899 +0000 UTC m=+828.296202839"
Oct 30 19:35:16.482637 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:16.481929    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4" podStartSLOduration=2.481903166 podStartE2EDuration="2.481903166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:35:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:35:16.475736098 +0000 UTC m=+828.318668037" watchObservedRunningTime="2026-10-30 19:35:16.481903166 +0000 UTC m=+828.324835168"
Oct 30 19:35:16.542892 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:16.536095    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl" podStartSLOduration=1.536071138 podStartE2EDuration="1.536071138s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:35:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:35:16.519323791 +0000 UTC m=+828.362255732" watchObservedRunningTime="2026-10-30 19:35:16.536071138 +0000 UTC m=+828.379003079"
Oct 30 19:35:16.570939 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:16.570909    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"
Oct 30 19:35:16.575099 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:16.575067    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"
Oct 30 19:35:16.576486 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:16.576459    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused" start-of-body=
Oct 30 19:35:16.576615 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:16.576510    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c" podUID="452bcb07-8aa6-4e70-8f3a-af8bfaf756a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused"
Oct 30 19:35:16.700888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:16.697071    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"
Oct 30 19:35:17.223780 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:17.219233    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"
Oct 30 19:35:17.229087 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:17.229035    5907 patch_prober.go:29] interesting pod/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 19:35:17.229367 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:17.229323    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4" podUID="27e1950d-b453-4f32-afd2-95ab9027a2e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 19:35:17.578988 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:17.577473    5907 patch_prober.go:29] interesting pod/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 19:35:17.578988 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:17.577543    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np" podUID="71285665-ae23-4783-a24f-535dccfb68c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 19:35:17.578988 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:17.578436    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused" start-of-body=
Oct 30 19:35:17.578988 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:17.578483    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c" podUID="452bcb07-8aa6-4e70-8f3a-af8bfaf756a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused"
Oct 30 19:35:17.695652 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:35:17.695535    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:35:17.700130 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:17.699999    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:35:17.700130 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:17.700060    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl" podUID="210e8e86-7046-4152-9a79-8f3c0fcdc900" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 19:35:18.215356 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:18.215314    5907 patch_prober.go:29] interesting pod/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 19:35:18.216145 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:18.215671    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4" podUID="27e1950d-b453-4f32-afd2-95ab9027a2e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 19:35:18.588984 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:18.584066    5907 patch_prober.go:29] interesting pod/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 19:35:18.588984 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:18.584138    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np" podUID="71285665-ae23-4783-a24f-535dccfb68c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 19:35:18.588984 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:18.585719    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused" start-of-body=
Oct 30 19:35:18.588984 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:18.585771    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c" podUID="452bcb07-8aa6-4e70-8f3a-af8bfaf756a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused"
Oct 30 19:35:18.698417 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:18.698376    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:35:18.698702 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:18.698652    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl" podUID="210e8e86-7046-4152-9a79-8f3c0fcdc900" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 19:35:19.255416 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:19.255376    5907 patch_prober.go:29] interesting pod/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 19:35:19.255721 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:19.255686    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4" podUID="27e1950d-b453-4f32-afd2-95ab9027a2e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 19:35:19.506685 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:19.506588    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6e50b175-l7ggb"]
Oct 30 19:35:19.590890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:19.538737    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e50b175-l7ggb"
Oct 30 19:35:19.596528 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:19.596492    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6e50b175-l7ggb"]
Oct 30 19:35:19.596982 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:19.596961    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:35:19.597266 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:19.597248    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:35:19.620829 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:19.610900    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e1415b6b-73cc-4d05-90c3-00d3df4b3188-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6e50b175-l7ggb\" (UID: \"e1415b6b-73cc-4d05-90c3-00d3df4b3188\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e50b175-l7ggb"
Oct 30 19:35:19.620829 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:19.610996    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e1415b6b-73cc-4d05-90c3-00d3df4b3188-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6e50b175-l7ggb\" (UID: \"e1415b6b-73cc-4d05-90c3-00d3df4b3188\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e50b175-l7ggb"
Oct 30 19:35:19.620829 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:19.611242    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4ztx\" (UniqueName: \"kubernetes.io/projected/e1415b6b-73cc-4d05-90c3-00d3df4b3188-kube-api-access-n4ztx\") pod \"maas-api-revoke-sub-6e50b175-l7ggb\" (UID: \"e1415b6b-73cc-4d05-90c3-00d3df4b3188\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e50b175-l7ggb"
Oct 30 19:35:19.643539 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:19.643500    5907 patch_prober.go:29] interesting pod/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 19:35:19.643841 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:19.643808    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np" podUID="71285665-ae23-4783-a24f-535dccfb68c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 19:35:19.729882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:19.722000    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:35:19.729882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:19.722081    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl" podUID="210e8e86-7046-4152-9a79-8f3c0fcdc900" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 19:35:19.743891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:19.742806    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused" start-of-body=
Oct 30 19:35:19.743891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:19.742897    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c" podUID="452bcb07-8aa6-4e70-8f3a-af8bfaf756a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused"
Oct 30 19:35:19.800002 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:19.799891    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4ztx\" (UniqueName: \"kubernetes.io/projected/e1415b6b-73cc-4d05-90c3-00d3df4b3188-kube-api-access-n4ztx\") pod \"maas-api-revoke-sub-6e50b175-l7ggb\" (UID: \"e1415b6b-73cc-4d05-90c3-00d3df4b3188\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e50b175-l7ggb"
Oct 30 19:35:19.809118 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:19.809075    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e1415b6b-73cc-4d05-90c3-00d3df4b3188-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6e50b175-l7ggb\" (UID: \"e1415b6b-73cc-4d05-90c3-00d3df4b3188\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e50b175-l7ggb"
Oct 30 19:35:19.818017 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:19.817977    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e1415b6b-73cc-4d05-90c3-00d3df4b3188-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6e50b175-l7ggb\" (UID: \"e1415b6b-73cc-4d05-90c3-00d3df4b3188\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e50b175-l7ggb"
Oct 30 19:35:19.835827 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:19.830329    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e1415b6b-73cc-4d05-90c3-00d3df4b3188-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6e50b175-l7ggb\" (UID: \"e1415b6b-73cc-4d05-90c3-00d3df4b3188\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e50b175-l7ggb"
Oct 30 19:35:19.844895 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:19.836211    5907 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:35:19.930889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:19.903345    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4ztx\" (UniqueName: \"kubernetes.io/projected/e1415b6b-73cc-4d05-90c3-00d3df4b3188-kube-api-access-n4ztx\") pod \"maas-api-revoke-sub-6e50b175-l7ggb\" (UID: \"e1415b6b-73cc-4d05-90c3-00d3df4b3188\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e50b175-l7ggb"
Oct 30 19:35:19.979943 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:19.971523    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:35:19.992524 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:19.992490    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e1415b6b-73cc-4d05-90c3-00d3df4b3188-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6e50b175-l7ggb\" (UID: \"e1415b6b-73cc-4d05-90c3-00d3df4b3188\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e50b175-l7ggb"
Oct 30 19:35:20.376165 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:20.359773    5907 patch_prober.go:29] interesting pod/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 19:35:20.376165 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:20.359915    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4" podUID="27e1950d-b453-4f32-afd2-95ab9027a2e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 19:35:20.418889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:20.418747    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e50b175-l7ggb"
Oct 30 19:35:20.712940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:20.646664    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused" start-of-body=
Oct 30 19:35:20.712940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:20.646758    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c" podUID="452bcb07-8aa6-4e70-8f3a-af8bfaf756a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused"
Oct 30 19:35:20.712940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:20.712328    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:35:20.712940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:20.712387    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl" podUID="210e8e86-7046-4152-9a79-8f3c0fcdc900" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 19:35:20.814835 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:20.796009    5907 patch_prober.go:29] interesting pod/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 19:35:20.814835 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:20.796067    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np" podUID="71285665-ae23-4783-a24f-535dccfb68c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 19:35:21.216284 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:21.216247    5907 patch_prober.go:29] interesting pod/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 19:35:21.216578 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:21.216552    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4" podUID="27e1950d-b453-4f32-afd2-95ab9027a2e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 19:35:21.604946 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:21.595668    5907 patch_prober.go:29] interesting pod/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 19:35:21.604946 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:21.595722    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np" podUID="71285665-ae23-4783-a24f-535dccfb68c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 19:35:21.604946 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:21.602318    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused" start-of-body=
Oct 30 19:35:21.604946 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:21.602370    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c" podUID="452bcb07-8aa6-4e70-8f3a-af8bfaf756a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused"
Oct 30 19:35:21.706887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:21.702425    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:35:21.706887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:21.702485    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl" podUID="210e8e86-7046-4152-9a79-8f3c0fcdc900" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 19:35:21.751895 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:21.745148    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 19:35:21.973565 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:21.953948    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:35:21.992578 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:21.992546    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6e50b175-l7ggb"]
Oct 30 19:35:22.223929 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:22.223381    5907 patch_prober.go:29] interesting pod/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 19:35:22.223929 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:22.223444    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4" podUID="27e1950d-b453-4f32-afd2-95ab9027a2e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 19:35:22.518351 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:22.518309    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e50b175-l7ggb" event={"ID":"e1415b6b-73cc-4d05-90c3-00d3df4b3188","Type":"ContainerStarted","Data":"728979c0757c433f7c00faaf826de63a87aa5edad5b12288d4cd94b837c421f4"}
Oct 30 19:35:22.573760 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:22.573286    5907 patch_prober.go:29] interesting pod/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 19:35:22.573760 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:22.573345    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np" podUID="71285665-ae23-4783-a24f-535dccfb68c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 19:35:22.596894 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:22.591068    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused" start-of-body=
Oct 30 19:35:22.596894 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:22.591131    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c" podUID="452bcb07-8aa6-4e70-8f3a-af8bfaf756a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused"
Oct 30 19:35:22.739305 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:22.697824    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:35:22.746358 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:22.745941    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl" podUID="210e8e86-7046-4152-9a79-8f3c0fcdc900" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 19:35:22.759384 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:22.755146    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:35:22.759384 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:22.755549    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:35:23.188269 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:22.965061    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:35:23.188269 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:23.026193    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:35:23.188269 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:23.063671    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": context deadline exceeded"
Oct 30 19:35:23.188269 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:23.064580    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:35:23.313886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:23.219161    5907 patch_prober.go:29] interesting pod/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 19:35:23.313886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:23.308013    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4" podUID="27e1950d-b453-4f32-afd2-95ab9027a2e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 19:35:23.712912 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:23.639943    5907 patch_prober.go:29] interesting pod/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 19:35:23.712912 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:23.640021    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np" podUID="71285665-ae23-4783-a24f-535dccfb68c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 19:35:23.712912 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:23.697198    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused" start-of-body=
Oct 30 19:35:23.712912 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:23.697260    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c" podUID="452bcb07-8aa6-4e70-8f3a-af8bfaf756a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused"
Oct 30 19:35:23.712912 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:23.710077    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:35:23.712912 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:23.710153    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl" podUID="210e8e86-7046-4152-9a79-8f3c0fcdc900" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 19:35:23.741416 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:23.733326    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:35:24.216472 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:24.216421    5907 patch_prober.go:29] interesting pod/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 19:35:24.217112 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:24.217072    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4" podUID="27e1950d-b453-4f32-afd2-95ab9027a2e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 19:35:24.576959 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:24.571840    5907 patch_prober.go:29] interesting pod/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 19:35:24.576959 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:24.571941    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np" podUID="71285665-ae23-4783-a24f-535dccfb68c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 19:35:24.576959 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:24.575915    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused" start-of-body=
Oct 30 19:35:24.576959 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:24.575971    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c" podUID="452bcb07-8aa6-4e70-8f3a-af8bfaf756a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused"
Oct 30 19:35:24.698388 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:24.697926    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:35:24.698388 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:24.697992    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl" podUID="210e8e86-7046-4152-9a79-8f3c0fcdc900" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 19:35:24.840107 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:24.839531    5907 generic.go:356] "Generic (PLEG): container finished" podID="e1415b6b-73cc-4d05-90c3-00d3df4b3188" containerID="1e8aef43e3c5d0eef98b32608a2f285a79c96935b99f73b6f9f90dcef681d2e0" exitCode=0
Oct 30 19:35:24.840107 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:24.839580    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e50b175-l7ggb" event={"ID":"e1415b6b-73cc-4d05-90c3-00d3df4b3188","Type":"ContainerDied","Data":"1e8aef43e3c5d0eef98b32608a2f285a79c96935b99f73b6f9f90dcef681d2e0"}
Oct 30 19:35:25.244883 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:25.238242    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"
Oct 30 19:35:25.244883 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:25.241518    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"
Oct 30 19:35:25.244883 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:25.242705    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"
Oct 30 19:35:25.584937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:25.578783    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"
Oct 30 19:35:25.584937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:25.580230    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"
Oct 30 19:35:25.598239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:25.592125    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"
Oct 30 19:35:25.610893 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:25.605329    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"
Oct 30 19:35:25.610893 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:25.608044    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"
Oct 30 19:35:25.622915 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:25.621824    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"
Oct 30 19:35:25.703310 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:25.703272    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"
Oct 30 19:35:25.703695 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:25.703672    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"
Oct 30 19:35:25.704380 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:25.704354    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"
Oct 30 19:35:26.046963 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:26.046935    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e50b175-l7ggb"
Oct 30 19:35:26.076758 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:26.076728    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e1415b6b-73cc-4d05-90c3-00d3df4b3188-kube-api-access-n4ztx\" (UniqueName: \"kubernetes.io/projected/e1415b6b-73cc-4d05-90c3-00d3df4b3188-kube-api-access-n4ztx\") pod \"e1415b6b-73cc-4d05-90c3-00d3df4b3188\" (UID: \"e1415b6b-73cc-4d05-90c3-00d3df4b3188\") "
Oct 30 19:35:26.076976 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:26.076951    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e1415b6b-73cc-4d05-90c3-00d3df4b3188-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e1415b6b-73cc-4d05-90c3-00d3df4b3188-maas-api-xks-ca\") pod \"e1415b6b-73cc-4d05-90c3-00d3df4b3188\" (UID: \"e1415b6b-73cc-4d05-90c3-00d3df4b3188\") "
Oct 30 19:35:26.077096 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:26.077080    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e1415b6b-73cc-4d05-90c3-00d3df4b3188-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e1415b6b-73cc-4d05-90c3-00d3df4b3188-maas-api-openshift-ca\") pod \"e1415b6b-73cc-4d05-90c3-00d3df4b3188\" (UID: \"e1415b6b-73cc-4d05-90c3-00d3df4b3188\") "
Oct 30 19:35:26.077545 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:26.077520    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1415b6b-73cc-4d05-90c3-00d3df4b3188-maas-api-openshift-ca" pod "e1415b6b-73cc-4d05-90c3-00d3df4b3188" (UID: "e1415b6b-73cc-4d05-90c3-00d3df4b3188"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:35:26.077767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:26.077749    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e1415b6b-73cc-4d05-90c3-00d3df4b3188-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:35:26.079372 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:26.079347    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1415b6b-73cc-4d05-90c3-00d3df4b3188-maas-api-xks-ca" pod "e1415b6b-73cc-4d05-90c3-00d3df4b3188" (UID: "e1415b6b-73cc-4d05-90c3-00d3df4b3188"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:35:26.079445 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:26.079419    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1415b6b-73cc-4d05-90c3-00d3df4b3188-kube-api-access-n4ztx" pod "e1415b6b-73cc-4d05-90c3-00d3df4b3188" (UID: "e1415b6b-73cc-4d05-90c3-00d3df4b3188"). InnerVolumeSpecName "kube-api-access-n4ztx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:35:26.179681 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:26.179648    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e1415b6b-73cc-4d05-90c3-00d3df4b3188-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:35:26.179681 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:26.179680    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4ztx\" (UniqueName: \"kubernetes.io/projected/e1415b6b-73cc-4d05-90c3-00d3df4b3188-kube-api-access-n4ztx\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:35:26.861142 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:26.861111    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e50b175-l7ggb"
Oct 30 19:35:26.861142 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:26.861121    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e50b175-l7ggb" event={"ID":"e1415b6b-73cc-4d05-90c3-00d3df4b3188","Type":"ContainerDied","Data":"728979c0757c433f7c00faaf826de63a87aa5edad5b12288d4cd94b837c421f4"}
Oct 30 19:35:26.861637 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:26.861212    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="728979c0757c433f7c00faaf826de63a87aa5edad5b12288d4cd94b837c421f4"
Oct 30 19:35:30.223146 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:30.223108    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:35:30.480386 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:30.480286    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"]
Oct 30 19:35:30.481454 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:30.481426    5907 cpu_manager.go:414] "containerMap: removing container" podUID="e1415b6b-73cc-4d05-90c3-00d3df4b3188" containerName="revoke-keys"
Oct 30 19:35:30.481454 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:30.481454    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e1415b6b-73cc-4d05-90c3-00d3df4b3188" containerName="revoke-keys"
Oct 30 19:35:30.481727 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:30.481622    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="e1415b6b-73cc-4d05-90c3-00d3df4b3188" containerName="revoke-keys"
Oct 30 19:35:30.511578 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:30.505239    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"
Oct 30 19:35:30.546999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:30.546951    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"]
Oct 30 19:35:30.571010 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:30.561154    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f90bc51f-a0ff-47f9-a334-1a2479c933fb-istio-token\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"
Oct 30 19:35:30.591550 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:30.585003    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f90bc51f-a0ff-47f9-a334-1a2479c933fb-workload-certs\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"
Oct 30 19:35:30.591550 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:30.585072    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfgl5\" (UniqueName: \"kubernetes.io/projected/f90bc51f-a0ff-47f9-a334-1a2479c933fb-kube-api-access-zfgl5\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"
Oct 30 19:35:30.591550 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:30.585322    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f90bc51f-a0ff-47f9-a334-1a2479c933fb-istio-podinfo\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"
Oct 30 19:35:30.591550 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:30.585479    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f90bc51f-a0ff-47f9-a334-1a2479c933fb-istio-data\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"
Oct 30 19:35:30.591550 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:30.585584    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f90bc51f-a0ff-47f9-a334-1a2479c933fb-credential-socket\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"
Oct 30 19:35:30.591550 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:30.585648    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f90bc51f-a0ff-47f9-a334-1a2479c933fb-workload-socket\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"
Oct 30 19:35:30.591550 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:30.586333    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f90bc51f-a0ff-47f9-a334-1a2479c933fb-istiod-ca-cert\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"
Oct 30 19:35:30.591550 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:30.586506    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f90bc51f-a0ff-47f9-a334-1a2479c933fb-istio-envoy\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"
Oct 30 19:35:30.687447 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:30.687414    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f90bc51f-a0ff-47f9-a334-1a2479c933fb-istiod-ca-cert\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"
Oct 30 19:35:30.687651 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:30.687506    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f90bc51f-a0ff-47f9-a334-1a2479c933fb-istio-envoy\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"
Oct 30 19:35:30.687651 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:30.687533    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f90bc51f-a0ff-47f9-a334-1a2479c933fb-istio-token\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"
Oct 30 19:35:30.687651 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:30.687563    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f90bc51f-a0ff-47f9-a334-1a2479c933fb-workload-certs\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"
Oct 30 19:35:30.687651 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:30.687600    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfgl5\" (UniqueName: \"kubernetes.io/projected/f90bc51f-a0ff-47f9-a334-1a2479c933fb-kube-api-access-zfgl5\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"
Oct 30 19:35:30.687916 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:30.687740    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f90bc51f-a0ff-47f9-a334-1a2479c933fb-istio-podinfo\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"
Oct 30 19:35:30.687916 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:30.687884    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f90bc51f-a0ff-47f9-a334-1a2479c933fb-istio-data\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"
Oct 30 19:35:30.688031 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:30.687975    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f90bc51f-a0ff-47f9-a334-1a2479c933fb-credential-socket\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"
Oct 30 19:35:30.688090 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:30.688036    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f90bc51f-a0ff-47f9-a334-1a2479c933fb-workload-socket\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"
Oct 30 19:35:30.688391 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:30.688182    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f90bc51f-a0ff-47f9-a334-1a2479c933fb-workload-certs\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"
Oct 30 19:35:30.688391 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:30.688311    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f90bc51f-a0ff-47f9-a334-1a2479c933fb-istio-data\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"
Oct 30 19:35:30.688576 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:30.688449    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f90bc51f-a0ff-47f9-a334-1a2479c933fb-workload-socket\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"
Oct 30 19:35:30.688576 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:30.688570    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f90bc51f-a0ff-47f9-a334-1a2479c933fb-credential-socket\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"
Oct 30 19:35:30.690544 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:30.690520    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f90bc51f-a0ff-47f9-a334-1a2479c933fb-istiod-ca-cert\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"
Oct 30 19:35:30.694446 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:30.691823    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f90bc51f-a0ff-47f9-a334-1a2479c933fb-istio-podinfo\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"
Oct 30 19:35:30.694446 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:30.693365    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f90bc51f-a0ff-47f9-a334-1a2479c933fb-istio-envoy\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"
Oct 30 19:35:30.709174 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:30.709145    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f90bc51f-a0ff-47f9-a334-1a2479c933fb-istio-token\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"
Oct 30 19:35:30.709318 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:30.709298    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfgl5\" (UniqueName: \"kubernetes.io/projected/f90bc51f-a0ff-47f9-a334-1a2479c933fb-kube-api-access-zfgl5\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"
Oct 30 19:35:30.925451 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:30.925419    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"
Oct 30 19:35:31.308889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:31.289684    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"]
Oct 30 19:35:31.334160 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:31.318795    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:35:31.334464 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:31.334430    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:35:31.334673 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:31.334653    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:35:31.358627 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:31.358590    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-305b2a9e-ssljn"]
Oct 30 19:35:31.394459 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:31.394429    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-305b2a9e-ssljn"
Oct 30 19:35:31.431885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:31.415091    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:35:31.431885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:31.415159    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:35:31.466908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:31.464288    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-305b2a9e-ssljn"]
Oct 30 19:35:31.525313 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:31.519109    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/41c4941d-9ed5-40d1-9f24-3ffb919eebeb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-305b2a9e-ssljn\" (UID: \"41c4941d-9ed5-40d1-9f24-3ffb919eebeb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-305b2a9e-ssljn"
Oct 30 19:35:31.525313 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:31.519205    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfddb\" (UniqueName: \"kubernetes.io/projected/41c4941d-9ed5-40d1-9f24-3ffb919eebeb-kube-api-access-gfddb\") pod \"maas-api-revoke-sub-305b2a9e-ssljn\" (UID: \"41c4941d-9ed5-40d1-9f24-3ffb919eebeb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-305b2a9e-ssljn"
Oct 30 19:35:31.525313 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:31.519368    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/41c4941d-9ed5-40d1-9f24-3ffb919eebeb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-305b2a9e-ssljn\" (UID: \"41c4941d-9ed5-40d1-9f24-3ffb919eebeb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-305b2a9e-ssljn"
Oct 30 19:35:31.622052 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:31.621988    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/41c4941d-9ed5-40d1-9f24-3ffb919eebeb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-305b2a9e-ssljn\" (UID: \"41c4941d-9ed5-40d1-9f24-3ffb919eebeb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-305b2a9e-ssljn"
Oct 30 19:35:31.622237 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:31.622126    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfddb\" (UniqueName: \"kubernetes.io/projected/41c4941d-9ed5-40d1-9f24-3ffb919eebeb-kube-api-access-gfddb\") pod \"maas-api-revoke-sub-305b2a9e-ssljn\" (UID: \"41c4941d-9ed5-40d1-9f24-3ffb919eebeb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-305b2a9e-ssljn"
Oct 30 19:35:31.622313 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:31.622295    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/41c4941d-9ed5-40d1-9f24-3ffb919eebeb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-305b2a9e-ssljn\" (UID: \"41c4941d-9ed5-40d1-9f24-3ffb919eebeb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-305b2a9e-ssljn"
Oct 30 19:35:31.623442 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:31.623256    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/41c4941d-9ed5-40d1-9f24-3ffb919eebeb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-305b2a9e-ssljn\" (UID: \"41c4941d-9ed5-40d1-9f24-3ffb919eebeb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-305b2a9e-ssljn"
Oct 30 19:35:31.626738 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:31.626715    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/41c4941d-9ed5-40d1-9f24-3ffb919eebeb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-305b2a9e-ssljn\" (UID: \"41c4941d-9ed5-40d1-9f24-3ffb919eebeb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-305b2a9e-ssljn"
Oct 30 19:35:31.674884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:31.672362    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfddb\" (UniqueName: \"kubernetes.io/projected/41c4941d-9ed5-40d1-9f24-3ffb919eebeb-kube-api-access-gfddb\") pod \"maas-api-revoke-sub-305b2a9e-ssljn\" (UID: \"41c4941d-9ed5-40d1-9f24-3ffb919eebeb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-305b2a9e-ssljn"
Oct 30 19:35:31.844231 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:31.843922    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-305b2a9e-ssljn"
Oct 30 19:35:31.943141 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:31.943103    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6" event={"ID":"f90bc51f-a0ff-47f9-a334-1a2479c933fb","Type":"ContainerStarted","Data":"7b9616a4c6ae3b1c8d4fa31c97cbc3ac8b49a47c6e5587696dc44603e63a8907"}
Oct 30 19:35:31.943504 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:31.943475    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6" event={"ID":"f90bc51f-a0ff-47f9-a334-1a2479c933fb","Type":"ContainerStarted","Data":"bb097d21dec36530abe58be0e6b8c0f0cf49663d8d285187ee0d7394645bfc16"}
Oct 30 19:35:32.026232 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:32.026154    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6" podStartSLOduration=2.026133244 podStartE2EDuration="2.026133244s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:35:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:35:32.025699278 +0000 UTC m=+843.868631231" watchObservedRunningTime="2026-10-30 19:35:32.026133244 +0000 UTC m=+843.869065186"
Oct 30 19:35:32.226348 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:32.226301    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-305b2a9e-ssljn"]
Oct 30 19:35:32.932618 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:32.932524    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"
Oct 30 19:35:32.933883 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:32.933831    5907 patch_prober.go:29] interesting pod/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 30 19:35:32.934126 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:32.934095    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6" podUID="f90bc51f-a0ff-47f9-a334-1a2479c933fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 30 19:35:32.993323 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:32.993279    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-305b2a9e-ssljn" event={"ID":"41c4941d-9ed5-40d1-9f24-3ffb919eebeb","Type":"ContainerStarted","Data":"d116edb9fac502126ee14e02503d8990590c55ef2d15607a55efa3d9b03fd7a0"}
Oct 30 19:35:32.993647 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:32.993619    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-305b2a9e-ssljn" event={"ID":"41c4941d-9ed5-40d1-9f24-3ffb919eebeb","Type":"ContainerStarted","Data":"c2f504b5f7a5914888f860f219e6f01d6e4fe64b1aaebd37d5099d5f3920693f"}
Oct 30 19:35:33.055236 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:33.055169    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-305b2a9e-ssljn" podStartSLOduration=2.055150485 podStartE2EDuration="2.055150485s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:35:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:35:33.039233559 +0000 UTC m=+844.882165498" watchObservedRunningTime="2026-10-30 19:35:33.055150485 +0000 UTC m=+844.898082425"
Oct 30 19:35:33.933928 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:33.933443    5907 patch_prober.go:29] interesting pod/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 30 19:35:33.933928 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:33.933515    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6" podUID="f90bc51f-a0ff-47f9-a334-1a2479c933fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 30 19:35:33.988575 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:33.988535    5907 generic.go:356] "Generic (PLEG): container finished" podID="41c4941d-9ed5-40d1-9f24-3ffb919eebeb" containerID="d116edb9fac502126ee14e02503d8990590c55ef2d15607a55efa3d9b03fd7a0" exitCode=0
Oct 30 19:35:33.988754 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:33.988619    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-305b2a9e-ssljn" event={"ID":"41c4941d-9ed5-40d1-9f24-3ffb919eebeb","Type":"ContainerDied","Data":"d116edb9fac502126ee14e02503d8990590c55ef2d15607a55efa3d9b03fd7a0"}
Oct 30 19:35:34.932987 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:34.932952    5907 patch_prober.go:29] interesting pod/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 30 19:35:34.933278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:34.933249    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6" podUID="f90bc51f-a0ff-47f9-a334-1a2479c933fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 30 19:35:35.235273 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:35.235091    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-305b2a9e-ssljn"
Oct 30 19:35:35.259749 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:35.259725    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/41c4941d-9ed5-40d1-9f24-3ffb919eebeb-kube-api-access-gfddb\" (UniqueName: \"kubernetes.io/projected/41c4941d-9ed5-40d1-9f24-3ffb919eebeb-kube-api-access-gfddb\") pod \"41c4941d-9ed5-40d1-9f24-3ffb919eebeb\" (UID: \"41c4941d-9ed5-40d1-9f24-3ffb919eebeb\") "
Oct 30 19:35:35.259928 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:35.259764    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/41c4941d-9ed5-40d1-9f24-3ffb919eebeb-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/41c4941d-9ed5-40d1-9f24-3ffb919eebeb-maas-api-xks-ca\") pod \"41c4941d-9ed5-40d1-9f24-3ffb919eebeb\" (UID: \"41c4941d-9ed5-40d1-9f24-3ffb919eebeb\") "
Oct 30 19:35:35.259928 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:35.259893    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/41c4941d-9ed5-40d1-9f24-3ffb919eebeb-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/41c4941d-9ed5-40d1-9f24-3ffb919eebeb-maas-api-openshift-ca\") pod \"41c4941d-9ed5-40d1-9f24-3ffb919eebeb\" (UID: \"41c4941d-9ed5-40d1-9f24-3ffb919eebeb\") "
Oct 30 19:35:35.260307 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:35.260280    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41c4941d-9ed5-40d1-9f24-3ffb919eebeb-maas-api-openshift-ca" pod "41c4941d-9ed5-40d1-9f24-3ffb919eebeb" (UID: "41c4941d-9ed5-40d1-9f24-3ffb919eebeb"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:35:35.260592 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:35.260567    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/41c4941d-9ed5-40d1-9f24-3ffb919eebeb-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:35:35.262265 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:35.262240    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41c4941d-9ed5-40d1-9f24-3ffb919eebeb-maas-api-xks-ca" pod "41c4941d-9ed5-40d1-9f24-3ffb919eebeb" (UID: "41c4941d-9ed5-40d1-9f24-3ffb919eebeb"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:35:35.262441 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:35.262426    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41c4941d-9ed5-40d1-9f24-3ffb919eebeb-kube-api-access-gfddb" pod "41c4941d-9ed5-40d1-9f24-3ffb919eebeb" (UID: "41c4941d-9ed5-40d1-9f24-3ffb919eebeb"). InnerVolumeSpecName "kube-api-access-gfddb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:35:35.362342 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:35.362303    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfddb\" (UniqueName: \"kubernetes.io/projected/41c4941d-9ed5-40d1-9f24-3ffb919eebeb-kube-api-access-gfddb\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:35:35.362489 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:35.362360    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/41c4941d-9ed5-40d1-9f24-3ffb919eebeb-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:35:35.931887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:35.926562    5907 patch_prober.go:29] interesting pod/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 30 19:35:35.931887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:35.926623    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6" podUID="f90bc51f-a0ff-47f9-a334-1a2479c933fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 30 19:35:36.005949 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:36.005917    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-305b2a9e-ssljn"
Oct 30 19:35:36.006107 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:36.005954    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-305b2a9e-ssljn" event={"ID":"41c4941d-9ed5-40d1-9f24-3ffb919eebeb","Type":"ContainerDied","Data":"c2f504b5f7a5914888f860f219e6f01d6e4fe64b1aaebd37d5099d5f3920693f"}
Oct 30 19:35:36.006107 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:36.006004    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c2f504b5f7a5914888f860f219e6f01d6e4fe64b1aaebd37d5099d5f3920693f"
Oct 30 19:35:36.935007 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:36.934968    5907 patch_prober.go:29] interesting pod/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 30 19:35:36.935484 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:36.935046    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6" podUID="f90bc51f-a0ff-47f9-a334-1a2479c933fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 30 19:35:37.556557 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:37.556527    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"]
Oct 30 19:35:37.557779 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:37.557754    5907 cpu_manager.go:414] "containerMap: removing container" podUID="41c4941d-9ed5-40d1-9f24-3ffb919eebeb" containerName="revoke-keys"
Oct 30 19:35:37.557779 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:37.557777    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="41c4941d-9ed5-40d1-9f24-3ffb919eebeb" containerName="revoke-keys"
Oct 30 19:35:37.558044 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:37.558029    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="41c4941d-9ed5-40d1-9f24-3ffb919eebeb" containerName="revoke-keys"
Oct 30 19:35:37.563418 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:37.563397    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"]
Oct 30 19:35:37.563542 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:37.563500    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:35:37.579666 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:37.579642    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-ef9a5a\"/\"openshift-service-ca.crt\""
Oct 30 19:35:37.579925 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:37.579903    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-ef9a5a\"/\"kube-root-ca.crt\""
Oct 30 19:35:37.592914 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:37.592850    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-ef9a5a\"/\"default-dockercfg-wtrg6\""
Oct 30 19:35:37.592914 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:37.592909    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-ef9a5a\"/\"distinct-2-w6-ef9a5a-kserve-self-signed-certs\""
Oct 30 19:35:37.598798 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:37.598727    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cklh9\" (UniqueName: \"kubernetes.io/projected/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-kube-api-access-cklh9\") pod \"distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx\" (UID: \"cf66ca70-f9e4-4248-9ae7-ee00390ce05a\") " pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:35:37.599258 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:37.598815    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-home\") pod \"distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx\" (UID: \"cf66ca70-f9e4-4248-9ae7-ee00390ce05a\") " pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:35:37.599258 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:37.598917    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-model-cache\") pod \"distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx\" (UID: \"cf66ca70-f9e4-4248-9ae7-ee00390ce05a\") " pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:35:37.599258 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:37.598942    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-dshm\") pod \"distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx\" (UID: \"cf66ca70-f9e4-4248-9ae7-ee00390ce05a\") " pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:35:37.599258 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:37.598999    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-tmp-dir\") pod \"distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx\" (UID: \"cf66ca70-f9e4-4248-9ae7-ee00390ce05a\") " pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:35:37.599258 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:37.599094    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-tls-certs\") pod \"distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx\" (UID: \"cf66ca70-f9e4-4248-9ae7-ee00390ce05a\") " pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:35:37.724889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:37.719518    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-tls-certs\") pod \"distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx\" (UID: \"cf66ca70-f9e4-4248-9ae7-ee00390ce05a\") " pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:35:37.724889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:37.719642    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cklh9\" (UniqueName: \"kubernetes.io/projected/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-kube-api-access-cklh9\") pod \"distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx\" (UID: \"cf66ca70-f9e4-4248-9ae7-ee00390ce05a\") " pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:35:37.724889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:37.719739    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-home\") pod \"distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx\" (UID: \"cf66ca70-f9e4-4248-9ae7-ee00390ce05a\") " pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:35:37.724889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:37.719891    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-model-cache\") pod \"distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx\" (UID: \"cf66ca70-f9e4-4248-9ae7-ee00390ce05a\") " pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:35:37.724889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:37.719938    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-dshm\") pod \"distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx\" (UID: \"cf66ca70-f9e4-4248-9ae7-ee00390ce05a\") " pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:35:37.724889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:37.720028    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-tmp-dir\") pod \"distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx\" (UID: \"cf66ca70-f9e4-4248-9ae7-ee00390ce05a\") " pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:35:37.724889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:37.720624    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-tmp-dir\") pod \"distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx\" (UID: \"cf66ca70-f9e4-4248-9ae7-ee00390ce05a\") " pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:35:37.724889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:37.722571    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-home\") pod \"distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx\" (UID: \"cf66ca70-f9e4-4248-9ae7-ee00390ce05a\") " pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:35:37.724889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:37.723221    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-model-cache\") pod \"distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx\" (UID: \"cf66ca70-f9e4-4248-9ae7-ee00390ce05a\") " pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:35:37.760457 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:37.754195    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-dshm\") pod \"distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx\" (UID: \"cf66ca70-f9e4-4248-9ae7-ee00390ce05a\") " pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:35:37.760457 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:37.754795    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-tls-certs\") pod \"distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx\" (UID: \"cf66ca70-f9e4-4248-9ae7-ee00390ce05a\") " pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:35:37.773895 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:37.772733    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cklh9\" (UniqueName: \"kubernetes.io/projected/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-kube-api-access-cklh9\") pod \"distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx\" (UID: \"cf66ca70-f9e4-4248-9ae7-ee00390ce05a\") " pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:35:37.881311 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:37.880141    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:35:37.932420 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:37.932380    5907 patch_prober.go:29] interesting pod/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 30 19:35:37.932686 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:37.932655    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6" podUID="f90bc51f-a0ff-47f9-a334-1a2479c933fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 30 19:35:38.295534 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:38.295505    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"]
Oct 30 19:35:38.296154 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:35:38.296131    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcf66ca70_f9e4_4248_9ae7_ee00390ce05a.slice/crio-52b3ebb7aa1cdb227486a19c314c85401a6e53999472ded555faac8b9ace16e8 WatchSource:0}: Error finding container 52b3ebb7aa1cdb227486a19c314c85401a6e53999472ded555faac8b9ace16e8: Status 404 returned error can't find the container with id 52b3ebb7aa1cdb227486a19c314c85401a6e53999472ded555faac8b9ace16e8
Oct 30 19:35:38.926790 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:38.926756    5907 patch_prober.go:29] interesting pod/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 30 19:35:38.927019 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:38.926805    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6" podUID="f90bc51f-a0ff-47f9-a334-1a2479c933fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 30 19:35:39.107672 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:39.107612    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" event={"ID":"cf66ca70-f9e4-4248-9ae7-ee00390ce05a","Type":"ContainerStarted","Data":"a8b2360b8d75055e61f5ff88e70e47605769fc66711280e2b5e3b4f7b146dd4a"}
Oct 30 19:35:39.109829 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:39.109801    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" event={"ID":"cf66ca70-f9e4-4248-9ae7-ee00390ce05a","Type":"ContainerStarted","Data":"52b3ebb7aa1cdb227486a19c314c85401a6e53999472ded555faac8b9ace16e8"}
Oct 30 19:35:39.131821 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:39.131777    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-af2965cf-tv9tb"]
Oct 30 19:35:39.143802 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:39.143769    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-af2965cf-tv9tb"
Oct 30 19:35:39.163110 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:39.163010    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:35:39.163265 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:39.163201    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:35:39.170051 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:39.170011    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-af2965cf-tv9tb"]
Oct 30 19:35:39.184269 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:39.184165    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podStartSLOduration=2.184144082 podStartE2EDuration="2.184144082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:35:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:35:39.172719756 +0000 UTC m=+851.015651698" watchObservedRunningTime="2026-10-30 19:35:39.184144082 +0000 UTC m=+851.027076024"
Oct 30 19:35:39.256328 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:39.256286    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/55c624df-a753-4418-9f4d-1e277205e7da-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-af2965cf-tv9tb\" (UID: \"55c624df-a753-4418-9f4d-1e277205e7da\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-af2965cf-tv9tb"
Oct 30 19:35:39.256531 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:39.256468    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx7qb\" (UniqueName: \"kubernetes.io/projected/55c624df-a753-4418-9f4d-1e277205e7da-kube-api-access-vx7qb\") pod \"maas-api-revoke-sub-af2965cf-tv9tb\" (UID: \"55c624df-a753-4418-9f4d-1e277205e7da\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-af2965cf-tv9tb"
Oct 30 19:35:39.256800 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:39.256776    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/55c624df-a753-4418-9f4d-1e277205e7da-maas-api-xks-ca\") pod \"maas-api-revoke-sub-af2965cf-tv9tb\" (UID: \"55c624df-a753-4418-9f4d-1e277205e7da\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-af2965cf-tv9tb"
Oct 30 19:35:39.368971 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:39.368931    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/55c624df-a753-4418-9f4d-1e277205e7da-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-af2965cf-tv9tb\" (UID: \"55c624df-a753-4418-9f4d-1e277205e7da\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-af2965cf-tv9tb"
Oct 30 19:35:39.369598 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:39.369571    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx7qb\" (UniqueName: \"kubernetes.io/projected/55c624df-a753-4418-9f4d-1e277205e7da-kube-api-access-vx7qb\") pod \"maas-api-revoke-sub-af2965cf-tv9tb\" (UID: \"55c624df-a753-4418-9f4d-1e277205e7da\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-af2965cf-tv9tb"
Oct 30 19:35:39.371379 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:39.371348    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/55c624df-a753-4418-9f4d-1e277205e7da-maas-api-xks-ca\") pod \"maas-api-revoke-sub-af2965cf-tv9tb\" (UID: \"55c624df-a753-4418-9f4d-1e277205e7da\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-af2965cf-tv9tb"
Oct 30 19:35:39.375653 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:39.370657    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/55c624df-a753-4418-9f4d-1e277205e7da-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-af2965cf-tv9tb\" (UID: \"55c624df-a753-4418-9f4d-1e277205e7da\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-af2965cf-tv9tb"
Oct 30 19:35:39.376366 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:39.376343    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/55c624df-a753-4418-9f4d-1e277205e7da-maas-api-xks-ca\") pod \"maas-api-revoke-sub-af2965cf-tv9tb\" (UID: \"55c624df-a753-4418-9f4d-1e277205e7da\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-af2965cf-tv9tb"
Oct 30 19:35:39.405247 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:39.405159    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx7qb\" (UniqueName: \"kubernetes.io/projected/55c624df-a753-4418-9f4d-1e277205e7da-kube-api-access-vx7qb\") pod \"maas-api-revoke-sub-af2965cf-tv9tb\" (UID: \"55c624df-a753-4418-9f4d-1e277205e7da\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-af2965cf-tv9tb"
Oct 30 19:35:39.475110 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:39.475032    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-af2965cf-tv9tb"
Oct 30 19:35:39.826015 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:35:39.825984    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55c624df_a753_4418_9f4d_1e277205e7da.slice/crio-b4ae55ac1737181adb6e0ce79ffb78cb2d9ff4faef3b58f9539bedcee233d5c1 WatchSource:0}: Error finding container b4ae55ac1737181adb6e0ce79ffb78cb2d9ff4faef3b58f9539bedcee233d5c1: Status 404 returned error can't find the container with id b4ae55ac1737181adb6e0ce79ffb78cb2d9ff4faef3b58f9539bedcee233d5c1
Oct 30 19:35:39.827133 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:39.827106    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-af2965cf-tv9tb"]
Oct 30 19:35:39.881476 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:39.881452    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:35:39.930528 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:39.930498    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"
Oct 30 19:35:39.930913 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:39.930887    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"
Oct 30 19:35:39.931629 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:39.931609    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"
Oct 30 19:35:40.121058 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:40.121009    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-af2965cf-tv9tb" event={"ID":"55c624df-a753-4418-9f4d-1e277205e7da","Type":"ContainerStarted","Data":"b4ae55ac1737181adb6e0ce79ffb78cb2d9ff4faef3b58f9539bedcee233d5c1"}
Oct 30 19:35:41.130944 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:41.130904    5907 generic.go:356] "Generic (PLEG): container finished" podID="55c624df-a753-4418-9f4d-1e277205e7da" containerID="482d8ba069499aee81689ba6fb159c0b5ce5ae88ec9a11600dc545d91d0cf4f1" exitCode=0
Oct 30 19:35:41.131475 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:41.130998    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-af2965cf-tv9tb" event={"ID":"55c624df-a753-4418-9f4d-1e277205e7da","Type":"ContainerDied","Data":"482d8ba069499aee81689ba6fb159c0b5ce5ae88ec9a11600dc545d91d0cf4f1"}
Oct 30 19:35:42.398275 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:42.398251    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-af2965cf-tv9tb"
Oct 30 19:35:42.439535 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:42.439507    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/55c624df-a753-4418-9f4d-1e277205e7da-kube-api-access-vx7qb\" (UniqueName: \"kubernetes.io/projected/55c624df-a753-4418-9f4d-1e277205e7da-kube-api-access-vx7qb\") pod \"55c624df-a753-4418-9f4d-1e277205e7da\" (UID: \"55c624df-a753-4418-9f4d-1e277205e7da\") "
Oct 30 19:35:42.440130 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:42.439843    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/55c624df-a753-4418-9f4d-1e277205e7da-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/55c624df-a753-4418-9f4d-1e277205e7da-maas-api-openshift-ca\") pod \"55c624df-a753-4418-9f4d-1e277205e7da\" (UID: \"55c624df-a753-4418-9f4d-1e277205e7da\") "
Oct 30 19:35:42.440130 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:42.439970    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/55c624df-a753-4418-9f4d-1e277205e7da-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/55c624df-a753-4418-9f4d-1e277205e7da-maas-api-xks-ca\") pod \"55c624df-a753-4418-9f4d-1e277205e7da\" (UID: \"55c624df-a753-4418-9f4d-1e277205e7da\") "
Oct 30 19:35:42.440444 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:42.440402    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55c624df-a753-4418-9f4d-1e277205e7da-maas-api-openshift-ca" pod "55c624df-a753-4418-9f4d-1e277205e7da" (UID: "55c624df-a753-4418-9f4d-1e277205e7da"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:35:42.440729 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:42.440709    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/55c624df-a753-4418-9f4d-1e277205e7da-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:35:42.442352 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:42.442291    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55c624df-a753-4418-9f4d-1e277205e7da-kube-api-access-vx7qb" pod "55c624df-a753-4418-9f4d-1e277205e7da" (UID: "55c624df-a753-4418-9f4d-1e277205e7da"). InnerVolumeSpecName "kube-api-access-vx7qb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:35:42.443506 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:42.443483    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55c624df-a753-4418-9f4d-1e277205e7da-maas-api-xks-ca" pod "55c624df-a753-4418-9f4d-1e277205e7da" (UID: "55c624df-a753-4418-9f4d-1e277205e7da"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:35:42.542628 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:42.542594    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vx7qb\" (UniqueName: \"kubernetes.io/projected/55c624df-a753-4418-9f4d-1e277205e7da-kube-api-access-vx7qb\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:35:42.542628 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:42.542623    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/55c624df-a753-4418-9f4d-1e277205e7da-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:35:43.148176 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:43.148145    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-af2965cf-tv9tb"
Oct 30 19:35:43.148176 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:43.148169    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-af2965cf-tv9tb" event={"ID":"55c624df-a753-4418-9f4d-1e277205e7da","Type":"ContainerDied","Data":"b4ae55ac1737181adb6e0ce79ffb78cb2d9ff4faef3b58f9539bedcee233d5c1"}
Oct 30 19:35:43.148400 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:43.148220    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b4ae55ac1737181adb6e0ce79ffb78cb2d9ff4faef3b58f9539bedcee233d5c1"
Oct 30 19:35:45.268090 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:45.268054    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"]
Oct 30 19:35:45.270848 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:45.270817    5907 cpu_manager.go:414] "containerMap: removing container" podUID="55c624df-a753-4418-9f4d-1e277205e7da" containerName="revoke-keys"
Oct 30 19:35:45.271069 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:45.271050    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55c624df-a753-4418-9f4d-1e277205e7da" containerName="revoke-keys"
Oct 30 19:35:45.271383 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:45.271364    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="55c624df-a753-4418-9f4d-1e277205e7da" containerName="revoke-keys"
Oct 30 19:35:45.373469 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:45.365460    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"
Oct 30 19:35:45.405882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:45.399881    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"]
Oct 30 19:35:45.495621 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:45.495587    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81eea74f-1fd9-41b0-8ecf-0501402c6a66-istiod-ca-cert\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"
Oct 30 19:35:45.495621 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:45.495627    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81eea74f-1fd9-41b0-8ecf-0501402c6a66-istio-data\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"
Oct 30 19:35:45.495924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:45.495666    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81eea74f-1fd9-41b0-8ecf-0501402c6a66-istio-envoy\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"
Oct 30 19:35:45.495924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:45.495872    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81eea74f-1fd9-41b0-8ecf-0501402c6a66-credential-socket\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"
Oct 30 19:35:45.496040 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:45.495959    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81eea74f-1fd9-41b0-8ecf-0501402c6a66-istio-podinfo\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"
Oct 30 19:35:45.496040 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:45.496018    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsr6j\" (UniqueName: \"kubernetes.io/projected/81eea74f-1fd9-41b0-8ecf-0501402c6a66-kube-api-access-fsr6j\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"
Oct 30 19:35:45.496138 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:45.496049    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81eea74f-1fd9-41b0-8ecf-0501402c6a66-workload-socket\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"
Oct 30 19:35:45.496191 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:45.496117    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81eea74f-1fd9-41b0-8ecf-0501402c6a66-istio-token\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"
Oct 30 19:35:45.496191 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:45.496179    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81eea74f-1fd9-41b0-8ecf-0501402c6a66-workload-certs\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"
Oct 30 19:35:45.597569 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:45.597494    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81eea74f-1fd9-41b0-8ecf-0501402c6a66-credential-socket\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"
Oct 30 19:35:45.597719 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:45.597577    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81eea74f-1fd9-41b0-8ecf-0501402c6a66-istio-podinfo\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"
Oct 30 19:35:45.597719 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:45.597624    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsr6j\" (UniqueName: \"kubernetes.io/projected/81eea74f-1fd9-41b0-8ecf-0501402c6a66-kube-api-access-fsr6j\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"
Oct 30 19:35:45.597719 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:45.597669    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81eea74f-1fd9-41b0-8ecf-0501402c6a66-workload-socket\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"
Oct 30 19:35:45.597719 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:45.597716    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81eea74f-1fd9-41b0-8ecf-0501402c6a66-istio-token\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"
Oct 30 19:35:45.597981 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:45.597763    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81eea74f-1fd9-41b0-8ecf-0501402c6a66-workload-certs\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"
Oct 30 19:35:45.597981 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:45.597851    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81eea74f-1fd9-41b0-8ecf-0501402c6a66-istiod-ca-cert\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"
Oct 30 19:35:45.597981 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:45.597895    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81eea74f-1fd9-41b0-8ecf-0501402c6a66-istio-data\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"
Oct 30 19:35:45.597981 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:45.597933    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81eea74f-1fd9-41b0-8ecf-0501402c6a66-credential-socket\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"
Oct 30 19:35:45.597981 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:45.597950    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81eea74f-1fd9-41b0-8ecf-0501402c6a66-istio-envoy\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"
Oct 30 19:35:45.598213 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:45.598119    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81eea74f-1fd9-41b0-8ecf-0501402c6a66-workload-socket\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"
Oct 30 19:35:45.598311 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:45.598275    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81eea74f-1fd9-41b0-8ecf-0501402c6a66-istio-data\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"
Oct 30 19:35:45.598611 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:45.598547    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81eea74f-1fd9-41b0-8ecf-0501402c6a66-istiod-ca-cert\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"
Oct 30 19:35:45.598754 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:45.598735    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81eea74f-1fd9-41b0-8ecf-0501402c6a66-workload-certs\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"
Oct 30 19:35:45.600327 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:45.600302    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81eea74f-1fd9-41b0-8ecf-0501402c6a66-istio-envoy\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"
Oct 30 19:35:45.600566 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:45.600544    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81eea74f-1fd9-41b0-8ecf-0501402c6a66-istio-podinfo\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"
Oct 30 19:35:45.618814 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:45.618780    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81eea74f-1fd9-41b0-8ecf-0501402c6a66-istio-token\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"
Oct 30 19:35:45.619006 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:45.618984    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsr6j\" (UniqueName: \"kubernetes.io/projected/81eea74f-1fd9-41b0-8ecf-0501402c6a66-kube-api-access-fsr6j\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"
Oct 30 19:35:45.721769 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:45.721728    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"
Oct 30 19:35:46.059361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:46.059312    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"]
Oct 30 19:35:46.065630 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:35:46.065602    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81eea74f_1fd9_41b0_8ecf_0501402c6a66.slice/crio-40b4917b5e42f9e099b8d7eb074d075de95321293d9b2490464781afa778b11a WatchSource:0}: Error finding container 40b4917b5e42f9e099b8d7eb074d075de95321293d9b2490464781afa778b11a: Status 404 returned error can't find the container with id 40b4917b5e42f9e099b8d7eb074d075de95321293d9b2490464781afa778b11a
Oct 30 19:35:46.070027 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:46.069988    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:35:46.070148 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:46.070058    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:35:46.070148 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:46.070087    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:35:46.178824 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:46.178758    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c" event={"ID":"81eea74f-1fd9-41b0-8ecf-0501402c6a66","Type":"ContainerStarted","Data":"5586215f406d71926acf24937a198f86cc4e4be5e19c8e9e111c314a4823a94d"}
Oct 30 19:35:46.178824 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:46.178827    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c" event={"ID":"81eea74f-1fd9-41b0-8ecf-0501402c6a66","Type":"ContainerStarted","Data":"40b4917b5e42f9e099b8d7eb074d075de95321293d9b2490464781afa778b11a"}
Oct 30 19:35:46.233223 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:46.233164    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c" podStartSLOduration=1.2331448 podStartE2EDuration="1.2331448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:35:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:35:46.230355507 +0000 UTC m=+858.073287446" watchObservedRunningTime="2026-10-30 19:35:46.2331448 +0000 UTC m=+858.076076740"
Oct 30 19:35:46.722334 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:46.722300    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"
Oct 30 19:35:47.503096 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:35:47.503063    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:35:47.728889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:47.723926    5907 patch_prober.go:29] interesting pod/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Oct 30 19:35:47.728889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:47.724000    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c" podUID="81eea74f-1fd9-41b0-8ecf-0501402c6a66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Oct 30 19:35:47.881039 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:47.881009    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:35:47.894114 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:47.894088    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:35:48.205259 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:48.205170    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:35:48.731884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:48.730997    5907 patch_prober.go:29] interesting pod/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Oct 30 19:35:48.731884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:48.731056    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c" podUID="81eea74f-1fd9-41b0-8ecf-0501402c6a66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Oct 30 19:35:49.722713 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:49.722245    5907 patch_prober.go:29] interesting pod/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Oct 30 19:35:49.722713 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:49.722308    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c" podUID="81eea74f-1fd9-41b0-8ecf-0501402c6a66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Oct 30 19:35:50.722919 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:50.722710    5907 patch_prober.go:29] interesting pod/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Oct 30 19:35:50.722919 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:50.722784    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c" podUID="81eea74f-1fd9-41b0-8ecf-0501402c6a66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Oct 30 19:35:50.733882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:50.733745    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7147304e-gk65b"]
Oct 30 19:35:50.750016 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:50.749989    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7147304e-gk65b"
Oct 30 19:35:50.751115 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:50.751092    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7147304e-gk65b"]
Oct 30 19:35:50.767471 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:50.767445    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:35:50.767804 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:50.767787    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:35:50.800943 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:50.800917    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/407d7b1c-719b-44ed-a746-e2ddbecd1798-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7147304e-gk65b\" (UID: \"407d7b1c-719b-44ed-a746-e2ddbecd1798\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7147304e-gk65b"
Oct 30 19:35:50.801225 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:50.801211    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/407d7b1c-719b-44ed-a746-e2ddbecd1798-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7147304e-gk65b\" (UID: \"407d7b1c-719b-44ed-a746-e2ddbecd1798\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7147304e-gk65b"
Oct 30 19:35:50.801395 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:50.801379    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9lmj\" (UniqueName: \"kubernetes.io/projected/407d7b1c-719b-44ed-a746-e2ddbecd1798-kube-api-access-p9lmj\") pod \"maas-api-revoke-sub-7147304e-gk65b\" (UID: \"407d7b1c-719b-44ed-a746-e2ddbecd1798\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7147304e-gk65b"
Oct 30 19:35:50.903514 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:50.903474    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/407d7b1c-719b-44ed-a746-e2ddbecd1798-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7147304e-gk65b\" (UID: \"407d7b1c-719b-44ed-a746-e2ddbecd1798\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7147304e-gk65b"
Oct 30 19:35:50.903997 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:50.903978    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/407d7b1c-719b-44ed-a746-e2ddbecd1798-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7147304e-gk65b\" (UID: \"407d7b1c-719b-44ed-a746-e2ddbecd1798\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7147304e-gk65b"
Oct 30 19:35:50.904100 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:50.904072    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9lmj\" (UniqueName: \"kubernetes.io/projected/407d7b1c-719b-44ed-a746-e2ddbecd1798-kube-api-access-p9lmj\") pod \"maas-api-revoke-sub-7147304e-gk65b\" (UID: \"407d7b1c-719b-44ed-a746-e2ddbecd1798\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7147304e-gk65b"
Oct 30 19:35:50.904620 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:50.904594    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/407d7b1c-719b-44ed-a746-e2ddbecd1798-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7147304e-gk65b\" (UID: \"407d7b1c-719b-44ed-a746-e2ddbecd1798\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7147304e-gk65b"
Oct 30 19:35:50.905998 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:50.905980    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/407d7b1c-719b-44ed-a746-e2ddbecd1798-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7147304e-gk65b\" (UID: \"407d7b1c-719b-44ed-a746-e2ddbecd1798\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7147304e-gk65b"
Oct 30 19:35:50.939083 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:50.939038    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9lmj\" (UniqueName: \"kubernetes.io/projected/407d7b1c-719b-44ed-a746-e2ddbecd1798-kube-api-access-p9lmj\") pod \"maas-api-revoke-sub-7147304e-gk65b\" (UID: \"407d7b1c-719b-44ed-a746-e2ddbecd1798\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7147304e-gk65b"
Oct 30 19:35:51.073853 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:51.073766    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7147304e-gk65b"
Oct 30 19:35:51.408722 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:35:51.408684    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod407d7b1c_719b_44ed_a746_e2ddbecd1798.slice/crio-ffed89be823396a2c4e327b758fdb7cd2f2e97a738d73b69b83bb856b0616108 WatchSource:0}: Error finding container ffed89be823396a2c4e327b758fdb7cd2f2e97a738d73b69b83bb856b0616108: Status 404 returned error can't find the container with id ffed89be823396a2c4e327b758fdb7cd2f2e97a738d73b69b83bb856b0616108
Oct 30 19:35:51.408722 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:51.408693    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7147304e-gk65b"]
Oct 30 19:35:51.730941 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:51.730217    5907 patch_prober.go:29] interesting pod/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Oct 30 19:35:51.730941 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:51.730277    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c" podUID="81eea74f-1fd9-41b0-8ecf-0501402c6a66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Oct 30 19:35:52.296759 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:52.296662    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7147304e-gk65b" event={"ID":"407d7b1c-719b-44ed-a746-e2ddbecd1798","Type":"ContainerStarted","Data":"7242303407d5a55d80f9b1cd760133fd57b2037a3ad30ea19810a6cc26c561fd"}
Oct 30 19:35:52.296759 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:52.296727    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7147304e-gk65b" event={"ID":"407d7b1c-719b-44ed-a746-e2ddbecd1798","Type":"ContainerStarted","Data":"ffed89be823396a2c4e327b758fdb7cd2f2e97a738d73b69b83bb856b0616108"}
Oct 30 19:35:52.341385 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:52.341331    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7147304e-gk65b" podStartSLOduration=2.3413169910000002 podStartE2EDuration="2.341316991s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:35:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:35:52.33758001 +0000 UTC m=+864.180511949" watchObservedRunningTime="2026-10-30 19:35:52.341316991 +0000 UTC m=+864.184248987"
Oct 30 19:35:52.722780 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:52.722749    5907 patch_prober.go:29] interesting pod/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Oct 30 19:35:52.722956 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:52.722810    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c" podUID="81eea74f-1fd9-41b0-8ecf-0501402c6a66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Oct 30 19:35:53.308800 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:53.308767    5907 generic.go:356] "Generic (PLEG): container finished" podID="407d7b1c-719b-44ed-a746-e2ddbecd1798" containerID="7242303407d5a55d80f9b1cd760133fd57b2037a3ad30ea19810a6cc26c561fd" exitCode=0
Oct 30 19:35:53.309225 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:53.308895    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7147304e-gk65b" event={"ID":"407d7b1c-719b-44ed-a746-e2ddbecd1798","Type":"ContainerDied","Data":"7242303407d5a55d80f9b1cd760133fd57b2037a3ad30ea19810a6cc26c561fd"}
Oct 30 19:35:53.734179 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:53.734145    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"
Oct 30 19:35:53.735187 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:53.735163    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"
Oct 30 19:35:53.739375 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:53.739347    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"
Oct 30 19:35:54.489733 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:54.489709    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7147304e-gk65b"
Oct 30 19:35:54.593065 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:54.592976    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/407d7b1c-719b-44ed-a746-e2ddbecd1798-kube-api-access-p9lmj\" (UniqueName: \"kubernetes.io/projected/407d7b1c-719b-44ed-a746-e2ddbecd1798-kube-api-access-p9lmj\") pod \"407d7b1c-719b-44ed-a746-e2ddbecd1798\" (UID: \"407d7b1c-719b-44ed-a746-e2ddbecd1798\") "
Oct 30 19:35:54.593240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:54.593064    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/407d7b1c-719b-44ed-a746-e2ddbecd1798-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/407d7b1c-719b-44ed-a746-e2ddbecd1798-maas-api-openshift-ca\") pod \"407d7b1c-719b-44ed-a746-e2ddbecd1798\" (UID: \"407d7b1c-719b-44ed-a746-e2ddbecd1798\") "
Oct 30 19:35:54.593240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:54.593222    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/407d7b1c-719b-44ed-a746-e2ddbecd1798-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/407d7b1c-719b-44ed-a746-e2ddbecd1798-maas-api-xks-ca\") pod \"407d7b1c-719b-44ed-a746-e2ddbecd1798\" (UID: \"407d7b1c-719b-44ed-a746-e2ddbecd1798\") "
Oct 30 19:35:54.593536 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:54.593508    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/407d7b1c-719b-44ed-a746-e2ddbecd1798-maas-api-openshift-ca" pod "407d7b1c-719b-44ed-a746-e2ddbecd1798" (UID: "407d7b1c-719b-44ed-a746-e2ddbecd1798"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:35:54.594249 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:54.594082    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/407d7b1c-719b-44ed-a746-e2ddbecd1798-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:35:54.595750 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:54.595719    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/407d7b1c-719b-44ed-a746-e2ddbecd1798-maas-api-xks-ca" pod "407d7b1c-719b-44ed-a746-e2ddbecd1798" (UID: "407d7b1c-719b-44ed-a746-e2ddbecd1798"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:35:54.595898 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:54.595814    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/407d7b1c-719b-44ed-a746-e2ddbecd1798-kube-api-access-p9lmj" pod "407d7b1c-719b-44ed-a746-e2ddbecd1798" (UID: "407d7b1c-719b-44ed-a746-e2ddbecd1798"). InnerVolumeSpecName "kube-api-access-p9lmj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:35:54.695911 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:54.695877    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/407d7b1c-719b-44ed-a746-e2ddbecd1798-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:35:54.695911 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:54.695911    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9lmj\" (UniqueName: \"kubernetes.io/projected/407d7b1c-719b-44ed-a746-e2ddbecd1798-kube-api-access-p9lmj\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:35:55.324964 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:55.324929    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7147304e-gk65b"
Oct 30 19:35:55.325165 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:55.324964    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7147304e-gk65b" event={"ID":"407d7b1c-719b-44ed-a746-e2ddbecd1798","Type":"ContainerDied","Data":"ffed89be823396a2c4e327b758fdb7cd2f2e97a738d73b69b83bb856b0616108"}
Oct 30 19:35:55.325165 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:55.325023    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ffed89be823396a2c4e327b758fdb7cd2f2e97a738d73b69b83bb856b0616108"
Oct 30 19:35:56.775358 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:56.775271    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-75645db4-blv48"]
Oct 30 19:35:56.777305 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:56.777275    5907 cpu_manager.go:414] "containerMap: removing container" podUID="407d7b1c-719b-44ed-a746-e2ddbecd1798" containerName="revoke-keys"
Oct 30 19:35:56.777494 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:56.777469    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="407d7b1c-719b-44ed-a746-e2ddbecd1798" containerName="revoke-keys"
Oct 30 19:35:56.777840 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:56.777823    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="407d7b1c-719b-44ed-a746-e2ddbecd1798" containerName="revoke-keys"
Oct 30 19:35:56.782786 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:56.782766    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-75645db4-blv48"
Oct 30 19:35:56.784055 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:56.784030    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-75645db4-blv48"]
Oct 30 19:35:56.800901 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:56.800082    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:35:56.800901 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:56.800152    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:35:56.851278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:56.851247    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cd39978b-2e78-405e-b806-a158b0c18c49-maas-api-xks-ca\") pod \"maas-api-revoke-sub-75645db4-blv48\" (UID: \"cd39978b-2e78-405e-b806-a158b0c18c49\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75645db4-blv48"
Oct 30 19:35:56.851439 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:56.851288    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cd39978b-2e78-405e-b806-a158b0c18c49-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-75645db4-blv48\" (UID: \"cd39978b-2e78-405e-b806-a158b0c18c49\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75645db4-blv48"
Oct 30 19:35:56.851439 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:56.851388    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csxnk\" (UniqueName: \"kubernetes.io/projected/cd39978b-2e78-405e-b806-a158b0c18c49-kube-api-access-csxnk\") pod \"maas-api-revoke-sub-75645db4-blv48\" (UID: \"cd39978b-2e78-405e-b806-a158b0c18c49\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75645db4-blv48"
Oct 30 19:35:56.952648 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:56.952616    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cd39978b-2e78-405e-b806-a158b0c18c49-maas-api-xks-ca\") pod \"maas-api-revoke-sub-75645db4-blv48\" (UID: \"cd39978b-2e78-405e-b806-a158b0c18c49\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75645db4-blv48"
Oct 30 19:35:56.952894 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:56.952664    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cd39978b-2e78-405e-b806-a158b0c18c49-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-75645db4-blv48\" (UID: \"cd39978b-2e78-405e-b806-a158b0c18c49\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75645db4-blv48"
Oct 30 19:35:56.952894 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:56.952725    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csxnk\" (UniqueName: \"kubernetes.io/projected/cd39978b-2e78-405e-b806-a158b0c18c49-kube-api-access-csxnk\") pod \"maas-api-revoke-sub-75645db4-blv48\" (UID: \"cd39978b-2e78-405e-b806-a158b0c18c49\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75645db4-blv48"
Oct 30 19:35:56.953447 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:56.953360    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cd39978b-2e78-405e-b806-a158b0c18c49-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-75645db4-blv48\" (UID: \"cd39978b-2e78-405e-b806-a158b0c18c49\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75645db4-blv48"
Oct 30 19:35:56.955170 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:56.955147    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cd39978b-2e78-405e-b806-a158b0c18c49-maas-api-xks-ca\") pod \"maas-api-revoke-sub-75645db4-blv48\" (UID: \"cd39978b-2e78-405e-b806-a158b0c18c49\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75645db4-blv48"
Oct 30 19:35:56.973386 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:56.973362    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-csxnk\" (UniqueName: \"kubernetes.io/projected/cd39978b-2e78-405e-b806-a158b0c18c49-kube-api-access-csxnk\") pod \"maas-api-revoke-sub-75645db4-blv48\" (UID: \"cd39978b-2e78-405e-b806-a158b0c18c49\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75645db4-blv48"
Oct 30 19:35:57.101820 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:57.101627    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-75645db4-blv48"
Oct 30 19:35:57.162360 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:57.162323    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-54f5f8a0-jd9kz"]
Oct 30 19:35:57.172578 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:57.172390    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-54f5f8a0-jd9kz"
Oct 30 19:35:57.199529 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:57.198294    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-54f5f8a0-jd9kz"]
Oct 30 19:35:57.255754 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:57.255714    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkhjq\" (UniqueName: \"kubernetes.io/projected/3f07cf47-62f3-425e-999a-fd496b530952-kube-api-access-fkhjq\") pod \"maas-api-revoke-sub-54f5f8a0-jd9kz\" (UID: \"3f07cf47-62f3-425e-999a-fd496b530952\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-54f5f8a0-jd9kz"
Oct 30 19:35:57.255973 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:57.255793    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3f07cf47-62f3-425e-999a-fd496b530952-maas-api-xks-ca\") pod \"maas-api-revoke-sub-54f5f8a0-jd9kz\" (UID: \"3f07cf47-62f3-425e-999a-fd496b530952\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-54f5f8a0-jd9kz"
Oct 30 19:35:57.255973 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:57.255894    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3f07cf47-62f3-425e-999a-fd496b530952-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-54f5f8a0-jd9kz\" (UID: \"3f07cf47-62f3-425e-999a-fd496b530952\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-54f5f8a0-jd9kz"
Oct 30 19:35:57.357123 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:57.357035    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkhjq\" (UniqueName: \"kubernetes.io/projected/3f07cf47-62f3-425e-999a-fd496b530952-kube-api-access-fkhjq\") pod \"maas-api-revoke-sub-54f5f8a0-jd9kz\" (UID: \"3f07cf47-62f3-425e-999a-fd496b530952\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-54f5f8a0-jd9kz"
Oct 30 19:35:57.357308 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:57.357141    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3f07cf47-62f3-425e-999a-fd496b530952-maas-api-xks-ca\") pod \"maas-api-revoke-sub-54f5f8a0-jd9kz\" (UID: \"3f07cf47-62f3-425e-999a-fd496b530952\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-54f5f8a0-jd9kz"
Oct 30 19:35:57.357308 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:57.357195    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3f07cf47-62f3-425e-999a-fd496b530952-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-54f5f8a0-jd9kz\" (UID: \"3f07cf47-62f3-425e-999a-fd496b530952\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-54f5f8a0-jd9kz"
Oct 30 19:35:57.357850 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:57.357826    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3f07cf47-62f3-425e-999a-fd496b530952-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-54f5f8a0-jd9kz\" (UID: \"3f07cf47-62f3-425e-999a-fd496b530952\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-54f5f8a0-jd9kz"
Oct 30 19:35:57.359911 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:57.359886    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3f07cf47-62f3-425e-999a-fd496b530952-maas-api-xks-ca\") pod \"maas-api-revoke-sub-54f5f8a0-jd9kz\" (UID: \"3f07cf47-62f3-425e-999a-fd496b530952\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-54f5f8a0-jd9kz"
Oct 30 19:35:57.377751 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:57.377728    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkhjq\" (UniqueName: \"kubernetes.io/projected/3f07cf47-62f3-425e-999a-fd496b530952-kube-api-access-fkhjq\") pod \"maas-api-revoke-sub-54f5f8a0-jd9kz\" (UID: \"3f07cf47-62f3-425e-999a-fd496b530952\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-54f5f8a0-jd9kz"
Oct 30 19:35:57.415892 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:35:57.415846    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd39978b_2e78_405e_b806_a158b0c18c49.slice/crio-8cb2ffcff067f5862624d26bd24acea5a58c9e815d30f565796dc3f2e4c2a696 WatchSource:0}: Error finding container 8cb2ffcff067f5862624d26bd24acea5a58c9e815d30f565796dc3f2e4c2a696: Status 404 returned error can't find the container with id 8cb2ffcff067f5862624d26bd24acea5a58c9e815d30f565796dc3f2e4c2a696
Oct 30 19:35:57.416586 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:57.416480    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-75645db4-blv48"]
Oct 30 19:35:57.513009 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:57.512980    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-54f5f8a0-jd9kz"
Oct 30 19:35:57.825690 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:57.825661    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-54f5f8a0-jd9kz"]
Oct 30 19:35:57.826454 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:35:57.826428    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f07cf47_62f3_425e_999a_fd496b530952.slice/crio-b9d8cd17733f9c31a469a670588f6d13c5d26832b582e63cae5822d126500d1e WatchSource:0}: Error finding container b9d8cd17733f9c31a469a670588f6d13c5d26832b582e63cae5822d126500d1e: Status 404 returned error can't find the container with id b9d8cd17733f9c31a469a670588f6d13c5d26832b582e63cae5822d126500d1e
Oct 30 19:35:58.361604 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:58.360971    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-54f5f8a0-jd9kz" event={"ID":"3f07cf47-62f3-425e-999a-fd496b530952","Type":"ContainerStarted","Data":"6e141defb52daa82e1325d196f68c7cbbe8ae957dcf67b6efa551375a15e98c7"}
Oct 30 19:35:58.361604 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:58.361021    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-54f5f8a0-jd9kz" event={"ID":"3f07cf47-62f3-425e-999a-fd496b530952","Type":"ContainerStarted","Data":"b9d8cd17733f9c31a469a670588f6d13c5d26832b582e63cae5822d126500d1e"}
Oct 30 19:35:58.363072 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:58.363045    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-75645db4-blv48" event={"ID":"cd39978b-2e78-405e-b806-a158b0c18c49","Type":"ContainerStarted","Data":"9e46ccb6e53e87ccc3262f30109ed8074ee0d6eda3297100fec31b09d089f8ed"}
Oct 30 19:35:58.363128 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:58.363098    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-75645db4-blv48" event={"ID":"cd39978b-2e78-405e-b806-a158b0c18c49","Type":"ContainerStarted","Data":"8cb2ffcff067f5862624d26bd24acea5a58c9e815d30f565796dc3f2e4c2a696"}
Oct 30 19:35:58.403925 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:58.403846    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-54f5f8a0-jd9kz" podStartSLOduration=1.403831592 podStartE2EDuration="1.403831592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:35:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:35:58.402823224 +0000 UTC m=+870.245755162" watchObservedRunningTime="2026-10-30 19:35:58.403831592 +0000 UTC m=+870.246763530"
Oct 30 19:35:58.450757 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:58.450709    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-75645db4-blv48" podStartSLOduration=2.450695892 podStartE2EDuration="2.450695892s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:35:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:35:58.447139863 +0000 UTC m=+870.290071802" watchObservedRunningTime="2026-10-30 19:35:58.450695892 +0000 UTC m=+870.293627831"
Oct 30 19:35:59.372310 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:59.372268    5907 generic.go:356] "Generic (PLEG): container finished" podID="3f07cf47-62f3-425e-999a-fd496b530952" containerID="6e141defb52daa82e1325d196f68c7cbbe8ae957dcf67b6efa551375a15e98c7" exitCode=0
Oct 30 19:35:59.372777 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:59.372317    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-54f5f8a0-jd9kz" event={"ID":"3f07cf47-62f3-425e-999a-fd496b530952","Type":"ContainerDied","Data":"6e141defb52daa82e1325d196f68c7cbbe8ae957dcf67b6efa551375a15e98c7"}
Oct 30 19:35:59.374269 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:59.374245    5907 generic.go:356] "Generic (PLEG): container finished" podID="cd39978b-2e78-405e-b806-a158b0c18c49" containerID="9e46ccb6e53e87ccc3262f30109ed8074ee0d6eda3297100fec31b09d089f8ed" exitCode=0
Oct 30 19:35:59.374385 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:35:59.374365    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-75645db4-blv48" event={"ID":"cd39978b-2e78-405e-b806-a158b0c18c49","Type":"ContainerDied","Data":"9e46ccb6e53e87ccc3262f30109ed8074ee0d6eda3297100fec31b09d089f8ed"}
Oct 30 19:36:01.075178 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:01.075046    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:01.813931 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:01.808964    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:01.912467 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:01.912419    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:02.232888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:02.222223    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 19:36:02.234024 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:02.233995    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:36:02.258429 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:02.242346    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:02.853888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:02.853334    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": context deadline exceeded"
Oct 30 19:36:02.940388 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:02.940334    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:03.243065 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:03.233967    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:03.652598 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:03.652542    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": context deadline exceeded"
Oct 30 19:36:03.770891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:03.756602    5907 patch_prober.go:29] interesting pod/metrics-server-6b87cd958c-prt2j container/metrics-server namespace/openshift-monitoring: Readiness probe status=failure output="Get \"https://10.134.0.14:10250/livez\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:36:03.854060 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:03.854006    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j" podUID="2ca12196-1929-4ece-a076-d36527b95ff1" containerName="metrics-server" probeResult="failure" output="Get \"https://10.134.0.14:10250/livez\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:03.903950 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:03.901481    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wd6m6" podUID="0b52bf40-f75b-465b-8947-10ea73874f85" containerName="main" probeResult="failure" output="Get \"https://10.134.0.37:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:03.930890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:03.904831    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:04.100813 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:04.029442    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 19:36:04.104707 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:04.037546    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-s826m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:36:04.104707 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:04.103100    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m" podUID="86efaf93-7f28-4e6a-9add-c8bdd87c5261" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:04.169985 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:36:04.158309    5907 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.728s"
Oct 30 19:36:04.247943 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:04.229964    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:36:04.247943 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:04.230043    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:36:04.247943 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:04.230106    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:36:04.247943 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:04.247718    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 19:36:04.303324 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:04.303261    5907 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:04.696890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:04.670912    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:04.718067 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:04.718021    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-75645db4-blv48"
Oct 30 19:36:04.856889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:04.856837    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd39978b-2e78-405e-b806-a158b0c18c49-kube-api-access-csxnk\" (UniqueName: \"kubernetes.io/projected/cd39978b-2e78-405e-b806-a158b0c18c49-kube-api-access-csxnk\") pod \"cd39978b-2e78-405e-b806-a158b0c18c49\" (UID: \"cd39978b-2e78-405e-b806-a158b0c18c49\") "
Oct 30 19:36:04.857230 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:04.857206    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cd39978b-2e78-405e-b806-a158b0c18c49-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cd39978b-2e78-405e-b806-a158b0c18c49-maas-api-xks-ca\") pod \"cd39978b-2e78-405e-b806-a158b0c18c49\" (UID: \"cd39978b-2e78-405e-b806-a158b0c18c49\") "
Oct 30 19:36:04.857399 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:04.857384    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cd39978b-2e78-405e-b806-a158b0c18c49-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cd39978b-2e78-405e-b806-a158b0c18c49-maas-api-openshift-ca\") pod \"cd39978b-2e78-405e-b806-a158b0c18c49\" (UID: \"cd39978b-2e78-405e-b806-a158b0c18c49\") "
Oct 30 19:36:04.864031 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:04.863998    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd39978b-2e78-405e-b806-a158b0c18c49-maas-api-openshift-ca" pod "cd39978b-2e78-405e-b806-a158b0c18c49" (UID: "cd39978b-2e78-405e-b806-a158b0c18c49"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:36:04.943889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:04.941225    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd39978b-2e78-405e-b806-a158b0c18c49-kube-api-access-csxnk" pod "cd39978b-2e78-405e-b806-a158b0c18c49" (UID: "cd39978b-2e78-405e-b806-a158b0c18c49"). InnerVolumeSpecName "kube-api-access-csxnk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:36:05.055675 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.021031    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd39978b-2e78-405e-b806-a158b0c18c49-maas-api-xks-ca" pod "cd39978b-2e78-405e-b806-a158b0c18c49" (UID: "cd39978b-2e78-405e-b806-a158b0c18c49"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:36:05.056148 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.022706    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-54f5f8a0-jd9kz"
Oct 30 19:36:05.056654 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.036966    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-54f5f8a0-jd9kz" event={"ID":"3f07cf47-62f3-425e-999a-fd496b530952","Type":"ContainerDied","Data":"b9d8cd17733f9c31a469a670588f6d13c5d26832b582e63cae5822d126500d1e"}
Oct 30 19:36:05.056821 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.056804    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b9d8cd17733f9c31a469a670588f6d13c5d26832b582e63cae5822d126500d1e"
Oct 30 19:36:05.056996 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.056974    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-75645db4-blv48" event={"ID":"cd39978b-2e78-405e-b806-a158b0c18c49","Type":"ContainerDied","Data":"8cb2ffcff067f5862624d26bd24acea5a58c9e815d30f565796dc3f2e4c2a696"}
Oct 30 19:36:05.057121 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.057106    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8cb2ffcff067f5862624d26bd24acea5a58c9e815d30f565796dc3f2e4c2a696"
Oct 30 19:36:05.057620 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.057592    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cd39978b-2e78-405e-b806-a158b0c18c49-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cd39978b-2e78-405e-b806-a158b0c18c49-maas-api-xks-ca\") pod \"cd39978b-2e78-405e-b806-a158b0c18c49\" (UID: \"cd39978b-2e78-405e-b806-a158b0c18c49\") "
Oct 30 19:36:05.058021 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:36:05.058005    5907 empty_dir.go:505] Warning: Unmount skipped because path does not exist: /var/lib/kubelet/pods/cd39978b-2e78-405e-b806-a158b0c18c49/volumes/kubernetes.io~secret/maas-api-xks-ca
Oct 30 19:36:05.058159 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.058141    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd39978b-2e78-405e-b806-a158b0c18c49-maas-api-xks-ca" pod "cd39978b-2e78-405e-b806-a158b0c18c49" (UID: "cd39978b-2e78-405e-b806-a158b0c18c49"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:36:05.125886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.101061    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:05.125886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.116759    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:05.249066 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.249023    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:05.293890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.292801    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-csxnk\" (UniqueName: \"kubernetes.io/projected/cd39978b-2e78-405e-b806-a158b0c18c49-kube-api-access-csxnk\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:05.293890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.292836    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cd39978b-2e78-405e-b806-a158b0c18c49-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:05.293890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.292851    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cd39978b-2e78-405e-b806-a158b0c18c49-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:05.403088 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.390227    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:36:05.410157 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.410118    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:36:05.411699 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.411675    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3f07cf47-62f3-425e-999a-fd496b530952-kube-api-access-fkhjq\" (UniqueName: \"kubernetes.io/projected/3f07cf47-62f3-425e-999a-fd496b530952-kube-api-access-fkhjq\") pod \"3f07cf47-62f3-425e-999a-fd496b530952\" (UID: \"3f07cf47-62f3-425e-999a-fd496b530952\") "
Oct 30 19:36:05.412005 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.411987    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3f07cf47-62f3-425e-999a-fd496b530952-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3f07cf47-62f3-425e-999a-fd496b530952-maas-api-xks-ca\") pod \"3f07cf47-62f3-425e-999a-fd496b530952\" (UID: \"3f07cf47-62f3-425e-999a-fd496b530952\") "
Oct 30 19:36:05.412633 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.412614    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3f07cf47-62f3-425e-999a-fd496b530952-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3f07cf47-62f3-425e-999a-fd496b530952-maas-api-openshift-ca\") pod \"3f07cf47-62f3-425e-999a-fd496b530952\" (UID: \"3f07cf47-62f3-425e-999a-fd496b530952\") "
Oct 30 19:36:05.448066 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.444646    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f07cf47-62f3-425e-999a-fd496b530952-maas-api-xks-ca" pod "3f07cf47-62f3-425e-999a-fd496b530952" (UID: "3f07cf47-62f3-425e-999a-fd496b530952"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:36:05.448066 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.447447    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f07cf47-62f3-425e-999a-fd496b530952-maas-api-openshift-ca" pod "3f07cf47-62f3-425e-999a-fd496b530952" (UID: "3f07cf47-62f3-425e-999a-fd496b530952"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:36:05.449435 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.449395    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3f07cf47-62f3-425e-999a-fd496b530952-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:05.499054 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.498982    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3f07cf47-62f3-425e-999a-fd496b530952-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:05.512169 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.510224    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f07cf47-62f3-425e-999a-fd496b530952-kube-api-access-fkhjq" pod "3f07cf47-62f3-425e-999a-fd496b530952" (UID: "3f07cf47-62f3-425e-999a-fd496b530952"). InnerVolumeSpecName "kube-api-access-fkhjq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:36:05.519015 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.518357    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:36:05.528954 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.528213    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:36:05.550902 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.547645    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6243ccc5-gjbng"]
Oct 30 19:36:05.550902 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.548638    5907 cpu_manager.go:414] "containerMap: removing container" podUID="3f07cf47-62f3-425e-999a-fd496b530952" containerName="revoke-keys"
Oct 30 19:36:05.550902 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.548655    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3f07cf47-62f3-425e-999a-fd496b530952" containerName="revoke-keys"
Oct 30 19:36:05.550902 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.548695    5907 cpu_manager.go:414] "containerMap: removing container" podUID="cd39978b-2e78-405e-b806-a158b0c18c49" containerName="revoke-keys"
Oct 30 19:36:05.550902 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.548702    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cd39978b-2e78-405e-b806-a158b0c18c49" containerName="revoke-keys"
Oct 30 19:36:05.550902 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.548906    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="3f07cf47-62f3-425e-999a-fd496b530952" containerName="revoke-keys"
Oct 30 19:36:05.550902 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.548923    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="cd39978b-2e78-405e-b806-a158b0c18c49" containerName="revoke-keys"
Oct 30 19:36:05.554463 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.553979    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6243ccc5-gjbng"
Oct 30 19:36:05.564728 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.564657    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6243ccc5-gjbng"]
Oct 30 19:36:05.602254 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.602215    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fkhjq\" (UniqueName: \"kubernetes.io/projected/3f07cf47-62f3-425e-999a-fd496b530952-kube-api-access-fkhjq\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:05.703246 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.703165    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1e44b517-ce16-497d-8c07-bc9a56e00e41-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6243ccc5-gjbng\" (UID: \"1e44b517-ce16-497d-8c07-bc9a56e00e41\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6243ccc5-gjbng"
Oct 30 19:36:05.703246 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.703228    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1e44b517-ce16-497d-8c07-bc9a56e00e41-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6243ccc5-gjbng\" (UID: \"1e44b517-ce16-497d-8c07-bc9a56e00e41\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6243ccc5-gjbng"
Oct 30 19:36:05.703460 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.703352    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wl5m9\" (UniqueName: \"kubernetes.io/projected/1e44b517-ce16-497d-8c07-bc9a56e00e41-kube-api-access-wl5m9\") pod \"maas-api-revoke-sub-6243ccc5-gjbng\" (UID: \"1e44b517-ce16-497d-8c07-bc9a56e00e41\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6243ccc5-gjbng"
Oct 30 19:36:05.822307 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.815380    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1e44b517-ce16-497d-8c07-bc9a56e00e41-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6243ccc5-gjbng\" (UID: \"1e44b517-ce16-497d-8c07-bc9a56e00e41\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6243ccc5-gjbng"
Oct 30 19:36:05.822307 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.815506    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1e44b517-ce16-497d-8c07-bc9a56e00e41-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6243ccc5-gjbng\" (UID: \"1e44b517-ce16-497d-8c07-bc9a56e00e41\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6243ccc5-gjbng"
Oct 30 19:36:05.822307 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.815559    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl5m9\" (UniqueName: \"kubernetes.io/projected/1e44b517-ce16-497d-8c07-bc9a56e00e41-kube-api-access-wl5m9\") pod \"maas-api-revoke-sub-6243ccc5-gjbng\" (UID: \"1e44b517-ce16-497d-8c07-bc9a56e00e41\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6243ccc5-gjbng"
Oct 30 19:36:05.822307 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.816823    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1e44b517-ce16-497d-8c07-bc9a56e00e41-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6243ccc5-gjbng\" (UID: \"1e44b517-ce16-497d-8c07-bc9a56e00e41\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6243ccc5-gjbng"
Oct 30 19:36:05.876341 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.857330    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1e44b517-ce16-497d-8c07-bc9a56e00e41-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6243ccc5-gjbng\" (UID: \"1e44b517-ce16-497d-8c07-bc9a56e00e41\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6243ccc5-gjbng"
Oct 30 19:36:05.932024 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.886238    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wl5m9\" (UniqueName: \"kubernetes.io/projected/1e44b517-ce16-497d-8c07-bc9a56e00e41-kube-api-access-wl5m9\") pod \"maas-api-revoke-sub-6243ccc5-gjbng\" (UID: \"1e44b517-ce16-497d-8c07-bc9a56e00e41\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6243ccc5-gjbng"
Oct 30 19:36:05.932024 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:05.888773    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6243ccc5-gjbng"
Oct 30 19:36:06.031074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:06.031036    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-54f5f8a0-jd9kz"
Oct 30 19:36:06.031269 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:06.031246    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-75645db4-blv48"
Oct 30 19:36:06.289107 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:36:06.289072    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e44b517_ce16_497d_8c07_bc9a56e00e41.slice/crio-23f61fe4ca153af21fae992afd596812b80b5abf29e834bdcc0593dea94c42fb WatchSource:0}: Error finding container 23f61fe4ca153af21fae992afd596812b80b5abf29e834bdcc0593dea94c42fb: Status 404 returned error can't find the container with id 23f61fe4ca153af21fae992afd596812b80b5abf29e834bdcc0593dea94c42fb
Oct 30 19:36:06.292544 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:06.292524    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6243ccc5-gjbng"]
Oct 30 19:36:07.040211 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:07.040118    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6243ccc5-gjbng" event={"ID":"1e44b517-ce16-497d-8c07-bc9a56e00e41","Type":"ContainerStarted","Data":"be564d018ebd236e70116021c8c401acc400a2b5974b30a689c0ae1a56573b5c"}
Oct 30 19:36:07.040368 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:07.040220    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6243ccc5-gjbng" event={"ID":"1e44b517-ce16-497d-8c07-bc9a56e00e41","Type":"ContainerStarted","Data":"23f61fe4ca153af21fae992afd596812b80b5abf29e834bdcc0593dea94c42fb"}
Oct 30 19:36:07.080514 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:07.080454    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6243ccc5-gjbng" podStartSLOduration=2.080437838 podStartE2EDuration="2.080437838s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:36:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:36:07.07976066 +0000 UTC m=+878.922692621" watchObservedRunningTime="2026-10-30 19:36:07.080437838 +0000 UTC m=+878.923369780"
Oct 30 19:36:08.050238 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:08.050197    5907 generic.go:356] "Generic (PLEG): container finished" podID="1e44b517-ce16-497d-8c07-bc9a56e00e41" containerID="be564d018ebd236e70116021c8c401acc400a2b5974b30a689c0ae1a56573b5c" exitCode=0
Oct 30 19:36:08.050670 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:08.050282    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6243ccc5-gjbng" event={"ID":"1e44b517-ce16-497d-8c07-bc9a56e00e41","Type":"ContainerDied","Data":"be564d018ebd236e70116021c8c401acc400a2b5974b30a689c0ae1a56573b5c"}
Oct 30 19:36:09.226018 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.225989    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6243ccc5-gjbng"
Oct 30 19:36:09.294968 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.294930    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-86a5394c-djb6k"]
Oct 30 19:36:09.296213 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.296189    5907 cpu_manager.go:414] "containerMap: removing container" podUID="1e44b517-ce16-497d-8c07-bc9a56e00e41" containerName="revoke-keys"
Oct 30 19:36:09.296383 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.296217    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1e44b517-ce16-497d-8c07-bc9a56e00e41" containerName="revoke-keys"
Oct 30 19:36:09.296481 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.296465    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="1e44b517-ce16-497d-8c07-bc9a56e00e41" containerName="revoke-keys"
Oct 30 19:36:09.301505 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.301468    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-86a5394c-djb6k"
Oct 30 19:36:09.330784 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.330757    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-86a5394c-djb6k"]
Oct 30 19:36:09.398588 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.398555    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1e44b517-ce16-497d-8c07-bc9a56e00e41-kube-api-access-wl5m9\" (UniqueName: \"kubernetes.io/projected/1e44b517-ce16-497d-8c07-bc9a56e00e41-kube-api-access-wl5m9\") pod \"1e44b517-ce16-497d-8c07-bc9a56e00e41\" (UID: \"1e44b517-ce16-497d-8c07-bc9a56e00e41\") "
Oct 30 19:36:09.398761 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.398661    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1e44b517-ce16-497d-8c07-bc9a56e00e41-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1e44b517-ce16-497d-8c07-bc9a56e00e41-maas-api-openshift-ca\") pod \"1e44b517-ce16-497d-8c07-bc9a56e00e41\" (UID: \"1e44b517-ce16-497d-8c07-bc9a56e00e41\") "
Oct 30 19:36:09.398761 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.398750    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1e44b517-ce16-497d-8c07-bc9a56e00e41-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1e44b517-ce16-497d-8c07-bc9a56e00e41-maas-api-xks-ca\") pod \"1e44b517-ce16-497d-8c07-bc9a56e00e41\" (UID: \"1e44b517-ce16-497d-8c07-bc9a56e00e41\") "
Oct 30 19:36:09.399160 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.399128    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e44b517-ce16-497d-8c07-bc9a56e00e41-maas-api-openshift-ca" pod "1e44b517-ce16-497d-8c07-bc9a56e00e41" (UID: "1e44b517-ce16-497d-8c07-bc9a56e00e41"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:36:09.399311 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.399292    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4b948304-5c98-48e7-ad5a-345322b06f95-maas-api-xks-ca\") pod \"maas-api-revoke-sub-86a5394c-djb6k\" (UID: \"4b948304-5c98-48e7-ad5a-345322b06f95\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-86a5394c-djb6k"
Oct 30 19:36:09.399378 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.399335    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76tmf\" (UniqueName: \"kubernetes.io/projected/4b948304-5c98-48e7-ad5a-345322b06f95-kube-api-access-76tmf\") pod \"maas-api-revoke-sub-86a5394c-djb6k\" (UID: \"4b948304-5c98-48e7-ad5a-345322b06f95\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-86a5394c-djb6k"
Oct 30 19:36:09.399586 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.399561    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4b948304-5c98-48e7-ad5a-345322b06f95-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-86a5394c-djb6k\" (UID: \"4b948304-5c98-48e7-ad5a-345322b06f95\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-86a5394c-djb6k"
Oct 30 19:36:09.399954 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.399929    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1e44b517-ce16-497d-8c07-bc9a56e00e41-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:09.401159 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.401136    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e44b517-ce16-497d-8c07-bc9a56e00e41-kube-api-access-wl5m9" pod "1e44b517-ce16-497d-8c07-bc9a56e00e41" (UID: "1e44b517-ce16-497d-8c07-bc9a56e00e41"). InnerVolumeSpecName "kube-api-access-wl5m9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:36:09.402006 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.401984    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e44b517-ce16-497d-8c07-bc9a56e00e41-maas-api-xks-ca" pod "1e44b517-ce16-497d-8c07-bc9a56e00e41" (UID: "1e44b517-ce16-497d-8c07-bc9a56e00e41"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:36:09.501253 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.501219    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4b948304-5c98-48e7-ad5a-345322b06f95-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-86a5394c-djb6k\" (UID: \"4b948304-5c98-48e7-ad5a-345322b06f95\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-86a5394c-djb6k"
Oct 30 19:36:09.501505 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.501479    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4b948304-5c98-48e7-ad5a-345322b06f95-maas-api-xks-ca\") pod \"maas-api-revoke-sub-86a5394c-djb6k\" (UID: \"4b948304-5c98-48e7-ad5a-345322b06f95\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-86a5394c-djb6k"
Oct 30 19:36:09.501586 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.501535    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76tmf\" (UniqueName: \"kubernetes.io/projected/4b948304-5c98-48e7-ad5a-345322b06f95-kube-api-access-76tmf\") pod \"maas-api-revoke-sub-86a5394c-djb6k\" (UID: \"4b948304-5c98-48e7-ad5a-345322b06f95\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-86a5394c-djb6k"
Oct 30 19:36:09.501699 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.501685    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wl5m9\" (UniqueName: \"kubernetes.io/projected/1e44b517-ce16-497d-8c07-bc9a56e00e41-kube-api-access-wl5m9\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:09.501754 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.501706    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1e44b517-ce16-497d-8c07-bc9a56e00e41-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:09.502035 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.502004    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4b948304-5c98-48e7-ad5a-345322b06f95-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-86a5394c-djb6k\" (UID: \"4b948304-5c98-48e7-ad5a-345322b06f95\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-86a5394c-djb6k"
Oct 30 19:36:09.504228 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.504202    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4b948304-5c98-48e7-ad5a-345322b06f95-maas-api-xks-ca\") pod \"maas-api-revoke-sub-86a5394c-djb6k\" (UID: \"4b948304-5c98-48e7-ad5a-345322b06f95\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-86a5394c-djb6k"
Oct 30 19:36:09.521846 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.521819    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76tmf\" (UniqueName: \"kubernetes.io/projected/4b948304-5c98-48e7-ad5a-345322b06f95-kube-api-access-76tmf\") pod \"maas-api-revoke-sub-86a5394c-djb6k\" (UID: \"4b948304-5c98-48e7-ad5a-345322b06f95\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-86a5394c-djb6k"
Oct 30 19:36:09.635794 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.635760    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-86a5394c-djb6k"
Oct 30 19:36:09.669368 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.669341    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f2c512a3-qh7tb"]
Oct 30 19:36:09.675268 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.675247    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2c512a3-qh7tb"
Oct 30 19:36:09.700912 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.695172    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f2c512a3-qh7tb"]
Oct 30 19:36:09.807208 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.807174    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e1c88445-d3b4-4fd8-9d68-98146be50d99-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f2c512a3-qh7tb\" (UID: \"e1c88445-d3b4-4fd8-9d68-98146be50d99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2c512a3-qh7tb"
Oct 30 19:36:09.807385 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.807350    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e1c88445-d3b4-4fd8-9d68-98146be50d99-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f2c512a3-qh7tb\" (UID: \"e1c88445-d3b4-4fd8-9d68-98146be50d99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2c512a3-qh7tb"
Oct 30 19:36:09.807440 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.807397    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8xjc\" (UniqueName: \"kubernetes.io/projected/e1c88445-d3b4-4fd8-9d68-98146be50d99-kube-api-access-p8xjc\") pod \"maas-api-revoke-sub-f2c512a3-qh7tb\" (UID: \"e1c88445-d3b4-4fd8-9d68-98146be50d99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2c512a3-qh7tb"
Oct 30 19:36:09.908950 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.908846    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e1c88445-d3b4-4fd8-9d68-98146be50d99-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f2c512a3-qh7tb\" (UID: \"e1c88445-d3b4-4fd8-9d68-98146be50d99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2c512a3-qh7tb"
Oct 30 19:36:09.908950 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.908922    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8xjc\" (UniqueName: \"kubernetes.io/projected/e1c88445-d3b4-4fd8-9d68-98146be50d99-kube-api-access-p8xjc\") pod \"maas-api-revoke-sub-f2c512a3-qh7tb\" (UID: \"e1c88445-d3b4-4fd8-9d68-98146be50d99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2c512a3-qh7tb"
Oct 30 19:36:09.909132 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.909090    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e1c88445-d3b4-4fd8-9d68-98146be50d99-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f2c512a3-qh7tb\" (UID: \"e1c88445-d3b4-4fd8-9d68-98146be50d99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2c512a3-qh7tb"
Oct 30 19:36:09.909536 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.909510    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e1c88445-d3b4-4fd8-9d68-98146be50d99-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f2c512a3-qh7tb\" (UID: \"e1c88445-d3b4-4fd8-9d68-98146be50d99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2c512a3-qh7tb"
Oct 30 19:36:09.911512 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.911491    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e1c88445-d3b4-4fd8-9d68-98146be50d99-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f2c512a3-qh7tb\" (UID: \"e1c88445-d3b4-4fd8-9d68-98146be50d99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2c512a3-qh7tb"
Oct 30 19:36:09.928951 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.928926    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8xjc\" (UniqueName: \"kubernetes.io/projected/e1c88445-d3b4-4fd8-9d68-98146be50d99-kube-api-access-p8xjc\") pod \"maas-api-revoke-sub-f2c512a3-qh7tb\" (UID: \"e1c88445-d3b4-4fd8-9d68-98146be50d99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2c512a3-qh7tb"
Oct 30 19:36:09.949583 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:36:09.949550    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b948304_5c98_48e7_ad5a_345322b06f95.slice/crio-96ef140dd6e207a72f86494425e721963d622aca1a389aeab42deb9562f0b9b4 WatchSource:0}: Error finding container 96ef140dd6e207a72f86494425e721963d622aca1a389aeab42deb9562f0b9b4: Status 404 returned error can't find the container with id 96ef140dd6e207a72f86494425e721963d622aca1a389aeab42deb9562f0b9b4
Oct 30 19:36:09.950363 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.950340    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-86a5394c-djb6k"]
Oct 30 19:36:09.993717 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:09.993695    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2c512a3-qh7tb"
Oct 30 19:36:10.082383 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:10.082351    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6243ccc5-gjbng"
Oct 30 19:36:10.082528 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:10.082386    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6243ccc5-gjbng" event={"ID":"1e44b517-ce16-497d-8c07-bc9a56e00e41","Type":"ContainerDied","Data":"23f61fe4ca153af21fae992afd596812b80b5abf29e834bdcc0593dea94c42fb"}
Oct 30 19:36:10.082528 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:10.082440    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="23f61fe4ca153af21fae992afd596812b80b5abf29e834bdcc0593dea94c42fb"
Oct 30 19:36:10.086419 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:10.086390    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-86a5394c-djb6k" event={"ID":"4b948304-5c98-48e7-ad5a-345322b06f95","Type":"ContainerStarted","Data":"96ef140dd6e207a72f86494425e721963d622aca1a389aeab42deb9562f0b9b4"}
Oct 30 19:36:10.323662 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:36:10.323629    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1c88445_d3b4_4fd8_9d68_98146be50d99.slice/crio-d6b4718fac426b5f9d547f121e659cb3255bb1ce4e999f7d1426473bae00df3a WatchSource:0}: Error finding container d6b4718fac426b5f9d547f121e659cb3255bb1ce4e999f7d1426473bae00df3a: Status 404 returned error can't find the container with id d6b4718fac426b5f9d547f121e659cb3255bb1ce4e999f7d1426473bae00df3a
Oct 30 19:36:10.324435 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:10.324401    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f2c512a3-qh7tb"]
Oct 30 19:36:11.096890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:11.096843    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2c512a3-qh7tb" event={"ID":"e1c88445-d3b4-4fd8-9d68-98146be50d99","Type":"ContainerStarted","Data":"b0a961f12815accaeb2507f32ef6cafaff6d25bb57f48a4a2c84f400789e4389"}
Oct 30 19:36:11.097107 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:11.096911    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2c512a3-qh7tb" event={"ID":"e1c88445-d3b4-4fd8-9d68-98146be50d99","Type":"ContainerStarted","Data":"d6b4718fac426b5f9d547f121e659cb3255bb1ce4e999f7d1426473bae00df3a"}
Oct 30 19:36:11.098730 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:11.098704    5907 generic.go:356] "Generic (PLEG): container finished" podID="4b948304-5c98-48e7-ad5a-345322b06f95" containerID="e8a3f7be371f4c05acd917a6d2a515e94b4d97118bf9c120364bf041d29e8463" exitCode=0
Oct 30 19:36:11.098873 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:11.098753    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-86a5394c-djb6k" event={"ID":"4b948304-5c98-48e7-ad5a-345322b06f95","Type":"ContainerDied","Data":"e8a3f7be371f4c05acd917a6d2a515e94b4d97118bf9c120364bf041d29e8463"}
Oct 30 19:36:11.143688 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:11.143636    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2c512a3-qh7tb" podStartSLOduration=2.143622019 podStartE2EDuration="2.143622019s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:36:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:36:11.13889697 +0000 UTC m=+882.981828904" watchObservedRunningTime="2026-10-30 19:36:11.143622019 +0000 UTC m=+882.986553983"
Oct 30 19:36:12.109725 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:12.109687    5907 generic.go:356] "Generic (PLEG): container finished" podID="e1c88445-d3b4-4fd8-9d68-98146be50d99" containerID="b0a961f12815accaeb2507f32ef6cafaff6d25bb57f48a4a2c84f400789e4389" exitCode=0
Oct 30 19:36:12.110242 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:12.109764    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2c512a3-qh7tb" event={"ID":"e1c88445-d3b4-4fd8-9d68-98146be50d99","Type":"ContainerDied","Data":"b0a961f12815accaeb2507f32ef6cafaff6d25bb57f48a4a2c84f400789e4389"}
Oct 30 19:36:12.316980 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:12.316951    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-86a5394c-djb6k"
Oct 30 19:36:12.457828 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:12.457730    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4b948304-5c98-48e7-ad5a-345322b06f95-kube-api-access-76tmf\" (UniqueName: \"kubernetes.io/projected/4b948304-5c98-48e7-ad5a-345322b06f95-kube-api-access-76tmf\") pod \"4b948304-5c98-48e7-ad5a-345322b06f95\" (UID: \"4b948304-5c98-48e7-ad5a-345322b06f95\") "
Oct 30 19:36:12.458026 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:12.457923    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4b948304-5c98-48e7-ad5a-345322b06f95-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4b948304-5c98-48e7-ad5a-345322b06f95-maas-api-xks-ca\") pod \"4b948304-5c98-48e7-ad5a-345322b06f95\" (UID: \"4b948304-5c98-48e7-ad5a-345322b06f95\") "
Oct 30 19:36:12.458097 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:12.458053    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4b948304-5c98-48e7-ad5a-345322b06f95-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4b948304-5c98-48e7-ad5a-345322b06f95-maas-api-openshift-ca\") pod \"4b948304-5c98-48e7-ad5a-345322b06f95\" (UID: \"4b948304-5c98-48e7-ad5a-345322b06f95\") "
Oct 30 19:36:12.458369 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:12.458347    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b948304-5c98-48e7-ad5a-345322b06f95-maas-api-openshift-ca" pod "4b948304-5c98-48e7-ad5a-345322b06f95" (UID: "4b948304-5c98-48e7-ad5a-345322b06f95"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:36:12.458817 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:12.458755    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4b948304-5c98-48e7-ad5a-345322b06f95-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:12.460090 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:12.460070    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b948304-5c98-48e7-ad5a-345322b06f95-kube-api-access-76tmf" pod "4b948304-5c98-48e7-ad5a-345322b06f95" (UID: "4b948304-5c98-48e7-ad5a-345322b06f95"). InnerVolumeSpecName "kube-api-access-76tmf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:36:12.460090 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:12.460079    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b948304-5c98-48e7-ad5a-345322b06f95-maas-api-xks-ca" pod "4b948304-5c98-48e7-ad5a-345322b06f95" (UID: "4b948304-5c98-48e7-ad5a-345322b06f95"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:36:12.559976 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:12.559949    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4b948304-5c98-48e7-ad5a-345322b06f95-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:12.559976 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:12.559972    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76tmf\" (UniqueName: \"kubernetes.io/projected/4b948304-5c98-48e7-ad5a-345322b06f95-kube-api-access-76tmf\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:13.120007 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:13.119977    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-86a5394c-djb6k"
Oct 30 19:36:13.120423 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:13.120010    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-86a5394c-djb6k" event={"ID":"4b948304-5c98-48e7-ad5a-345322b06f95","Type":"ContainerDied","Data":"96ef140dd6e207a72f86494425e721963d622aca1a389aeab42deb9562f0b9b4"}
Oct 30 19:36:13.120423 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:13.120071    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="96ef140dd6e207a72f86494425e721963d622aca1a389aeab42deb9562f0b9b4"
Oct 30 19:36:13.283799 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:13.283776    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2c512a3-qh7tb"
Oct 30 19:36:13.372274 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:13.372174    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e1c88445-d3b4-4fd8-9d68-98146be50d99-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e1c88445-d3b4-4fd8-9d68-98146be50d99-maas-api-xks-ca\") pod \"e1c88445-d3b4-4fd8-9d68-98146be50d99\" (UID: \"e1c88445-d3b4-4fd8-9d68-98146be50d99\") "
Oct 30 19:36:13.372455 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:13.372373    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e1c88445-d3b4-4fd8-9d68-98146be50d99-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e1c88445-d3b4-4fd8-9d68-98146be50d99-maas-api-openshift-ca\") pod \"e1c88445-d3b4-4fd8-9d68-98146be50d99\" (UID: \"e1c88445-d3b4-4fd8-9d68-98146be50d99\") "
Oct 30 19:36:13.372613 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:13.372589    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e1c88445-d3b4-4fd8-9d68-98146be50d99-kube-api-access-p8xjc\" (UniqueName: \"kubernetes.io/projected/e1c88445-d3b4-4fd8-9d68-98146be50d99-kube-api-access-p8xjc\") pod \"e1c88445-d3b4-4fd8-9d68-98146be50d99\" (UID: \"e1c88445-d3b4-4fd8-9d68-98146be50d99\") "
Oct 30 19:36:13.372806 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:13.372774    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1c88445-d3b4-4fd8-9d68-98146be50d99-maas-api-openshift-ca" pod "e1c88445-d3b4-4fd8-9d68-98146be50d99" (UID: "e1c88445-d3b4-4fd8-9d68-98146be50d99"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:36:13.373508 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:13.373426    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e1c88445-d3b4-4fd8-9d68-98146be50d99-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:13.374999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:13.374973    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1c88445-d3b4-4fd8-9d68-98146be50d99-maas-api-xks-ca" pod "e1c88445-d3b4-4fd8-9d68-98146be50d99" (UID: "e1c88445-d3b4-4fd8-9d68-98146be50d99"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:36:13.375141 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:13.375079    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1c88445-d3b4-4fd8-9d68-98146be50d99-kube-api-access-p8xjc" pod "e1c88445-d3b4-4fd8-9d68-98146be50d99" (UID: "e1c88445-d3b4-4fd8-9d68-98146be50d99"). InnerVolumeSpecName "kube-api-access-p8xjc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:36:13.474579 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:13.474536    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8xjc\" (UniqueName: \"kubernetes.io/projected/e1c88445-d3b4-4fd8-9d68-98146be50d99-kube-api-access-p8xjc\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:13.474579 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:13.474569    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e1c88445-d3b4-4fd8-9d68-98146be50d99-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:14.129772 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:14.129738    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2c512a3-qh7tb"
Oct 30 19:36:14.130215 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:14.129785    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f2c512a3-qh7tb" event={"ID":"e1c88445-d3b4-4fd8-9d68-98146be50d99","Type":"ContainerDied","Data":"d6b4718fac426b5f9d547f121e659cb3255bb1ce4e999f7d1426473bae00df3a"}
Oct 30 19:36:14.130215 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:14.129833    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d6b4718fac426b5f9d547f121e659cb3255bb1ce4e999f7d1426473bae00df3a"
Oct 30 19:36:16.094645 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.094607    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"]
Oct 30 19:36:16.095347 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.095333    5907 cpu_manager.go:414] "containerMap: removing container" podUID="e1c88445-d3b4-4fd8-9d68-98146be50d99" containerName="revoke-keys"
Oct 30 19:36:16.095405 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.095349    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e1c88445-d3b4-4fd8-9d68-98146be50d99" containerName="revoke-keys"
Oct 30 19:36:16.095405 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.095387    5907 cpu_manager.go:414] "containerMap: removing container" podUID="4b948304-5c98-48e7-ad5a-345322b06f95" containerName="revoke-keys"
Oct 30 19:36:16.095405 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.095392    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4b948304-5c98-48e7-ad5a-345322b06f95" containerName="revoke-keys"
Oct 30 19:36:16.095503 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.095494    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="e1c88445-d3b4-4fd8-9d68-98146be50d99" containerName="revoke-keys"
Oct 30 19:36:16.095538 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.095509    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="4b948304-5c98-48e7-ad5a-345322b06f95" containerName="revoke-keys"
Oct 30 19:36:16.099900 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.099855    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"
Oct 30 19:36:16.122580 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.122539    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34a1c006-cd05-48d7-9cb1-00635b1f4729-istiod-ca-cert\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"
Oct 30 19:36:16.122758 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.122737    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34a1c006-cd05-48d7-9cb1-00635b1f4729-istio-token\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"
Oct 30 19:36:16.122833 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.122795    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34a1c006-cd05-48d7-9cb1-00635b1f4729-workload-socket\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"
Oct 30 19:36:16.122899 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.122885    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34a1c006-cd05-48d7-9cb1-00635b1f4729-credential-socket\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"
Oct 30 19:36:16.122954 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.122937    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34a1c006-cd05-48d7-9cb1-00635b1f4729-workload-certs\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"
Oct 30 19:36:16.123088 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.123065    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx7kh\" (UniqueName: \"kubernetes.io/projected/34a1c006-cd05-48d7-9cb1-00635b1f4729-kube-api-access-xx7kh\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"
Oct 30 19:36:16.123152 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.123123    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34a1c006-cd05-48d7-9cb1-00635b1f4729-istio-podinfo\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"
Oct 30 19:36:16.123287 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.123270    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34a1c006-cd05-48d7-9cb1-00635b1f4729-istio-envoy\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"
Oct 30 19:36:16.123344 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.123328    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34a1c006-cd05-48d7-9cb1-00635b1f4729-istio-data\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"
Oct 30 19:36:16.136922 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.136889    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"]
Oct 30 19:36:16.224044 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.224011    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4202bb04-nn2r6"]
Oct 30 19:36:16.224337 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.224312    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34a1c006-cd05-48d7-9cb1-00635b1f4729-istio-token\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"
Oct 30 19:36:16.224403 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.224372    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34a1c006-cd05-48d7-9cb1-00635b1f4729-workload-socket\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"
Oct 30 19:36:16.224442 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.224419    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34a1c006-cd05-48d7-9cb1-00635b1f4729-credential-socket\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"
Oct 30 19:36:16.224496 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.224464    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34a1c006-cd05-48d7-9cb1-00635b1f4729-workload-certs\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"
Oct 30 19:36:16.224551 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.224526    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx7kh\" (UniqueName: \"kubernetes.io/projected/34a1c006-cd05-48d7-9cb1-00635b1f4729-kube-api-access-xx7kh\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"
Oct 30 19:36:16.224677 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.224653    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34a1c006-cd05-48d7-9cb1-00635b1f4729-istio-podinfo\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"
Oct 30 19:36:16.224894 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.224841    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34a1c006-cd05-48d7-9cb1-00635b1f4729-credential-socket\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"
Oct 30 19:36:16.225024 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.224844    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34a1c006-cd05-48d7-9cb1-00635b1f4729-istio-envoy\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"
Oct 30 19:36:16.225024 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.224969    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34a1c006-cd05-48d7-9cb1-00635b1f4729-workload-socket\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"
Oct 30 19:36:16.225024 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.224976    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34a1c006-cd05-48d7-9cb1-00635b1f4729-istio-data\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"
Oct 30 19:36:16.225024 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.224982    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34a1c006-cd05-48d7-9cb1-00635b1f4729-workload-certs\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"
Oct 30 19:36:16.225024 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.225020    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34a1c006-cd05-48d7-9cb1-00635b1f4729-istiod-ca-cert\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"
Oct 30 19:36:16.225284 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.225256    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34a1c006-cd05-48d7-9cb1-00635b1f4729-istio-data\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"
Oct 30 19:36:16.225951 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.225928    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34a1c006-cd05-48d7-9cb1-00635b1f4729-istiod-ca-cert\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"
Oct 30 19:36:16.227366 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.227339    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34a1c006-cd05-48d7-9cb1-00635b1f4729-istio-envoy\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"
Oct 30 19:36:16.227677 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.227653    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34a1c006-cd05-48d7-9cb1-00635b1f4729-istio-podinfo\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"
Oct 30 19:36:16.229683 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.229663    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4202bb04-nn2r6"
Oct 30 19:36:16.245741 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.245718    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:36:16.246214 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.246189    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:36:16.248008 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.247982    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx7kh\" (UniqueName: \"kubernetes.io/projected/34a1c006-cd05-48d7-9cb1-00635b1f4729-kube-api-access-xx7kh\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"
Oct 30 19:36:16.248897 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.248838    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34a1c006-cd05-48d7-9cb1-00635b1f4729-istio-token\") pod \"e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") " pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"
Oct 30 19:36:16.249290 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.249268    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4202bb04-nn2r6"]
Oct 30 19:36:16.326959 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.326914    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/27c3c568-8107-46bf-8c4a-c35d7b549e4b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4202bb04-nn2r6\" (UID: \"27c3c568-8107-46bf-8c4a-c35d7b549e4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4202bb04-nn2r6"
Oct 30 19:36:16.327102 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.326992    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzgvj\" (UniqueName: \"kubernetes.io/projected/27c3c568-8107-46bf-8c4a-c35d7b549e4b-kube-api-access-lzgvj\") pod \"maas-api-revoke-sub-4202bb04-nn2r6\" (UID: \"27c3c568-8107-46bf-8c4a-c35d7b549e4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4202bb04-nn2r6"
Oct 30 19:36:16.327178 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.327104    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/27c3c568-8107-46bf-8c4a-c35d7b549e4b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4202bb04-nn2r6\" (UID: \"27c3c568-8107-46bf-8c4a-c35d7b549e4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4202bb04-nn2r6"
Oct 30 19:36:16.414532 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.414504    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"
Oct 30 19:36:16.433574 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.433538    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/27c3c568-8107-46bf-8c4a-c35d7b549e4b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4202bb04-nn2r6\" (UID: \"27c3c568-8107-46bf-8c4a-c35d7b549e4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4202bb04-nn2r6"
Oct 30 19:36:16.433740 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.433615    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzgvj\" (UniqueName: \"kubernetes.io/projected/27c3c568-8107-46bf-8c4a-c35d7b549e4b-kube-api-access-lzgvj\") pod \"maas-api-revoke-sub-4202bb04-nn2r6\" (UID: \"27c3c568-8107-46bf-8c4a-c35d7b549e4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4202bb04-nn2r6"
Oct 30 19:36:16.433740 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.433672    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/27c3c568-8107-46bf-8c4a-c35d7b549e4b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4202bb04-nn2r6\" (UID: \"27c3c568-8107-46bf-8c4a-c35d7b549e4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4202bb04-nn2r6"
Oct 30 19:36:16.434512 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.434479    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/27c3c568-8107-46bf-8c4a-c35d7b549e4b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4202bb04-nn2r6\" (UID: \"27c3c568-8107-46bf-8c4a-c35d7b549e4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4202bb04-nn2r6"
Oct 30 19:36:16.436018 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.435990    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/27c3c568-8107-46bf-8c4a-c35d7b549e4b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4202bb04-nn2r6\" (UID: \"27c3c568-8107-46bf-8c4a-c35d7b549e4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4202bb04-nn2r6"
Oct 30 19:36:16.455166 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.455129    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzgvj\" (UniqueName: \"kubernetes.io/projected/27c3c568-8107-46bf-8c4a-c35d7b549e4b-kube-api-access-lzgvj\") pod \"maas-api-revoke-sub-4202bb04-nn2r6\" (UID: \"27c3c568-8107-46bf-8c4a-c35d7b549e4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4202bb04-nn2r6"
Oct 30 19:36:16.587924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.587890    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4202bb04-nn2r6"
Oct 30 19:36:16.749963 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:36:16.749931    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34a1c006_cd05_48d7_9cb1_00635b1f4729.slice/crio-82b3736a0f51ffa837c8ce1432735316682b2938a4aee208ad615eacd85f2e3b WatchSource:0}: Error finding container 82b3736a0f51ffa837c8ce1432735316682b2938a4aee208ad615eacd85f2e3b: Status 404 returned error can't find the container with id 82b3736a0f51ffa837c8ce1432735316682b2938a4aee208ad615eacd85f2e3b
Oct 30 19:36:16.750987 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.750966    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"]
Oct 30 19:36:16.752817 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.752782    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:36:16.752934 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.752851    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:36:16.752983 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.752933    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:36:16.926062 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:16.924177    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4202bb04-nn2r6"]
Oct 30 19:36:17.196061 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:17.196021    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc" event={"ID":"34a1c006-cd05-48d7-9cb1-00635b1f4729","Type":"ContainerStarted","Data":"094132593af5cd182cda41d836026dc341d4ec3a843350b75271762cf774f64f"}
Oct 30 19:36:17.196526 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:17.196091    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc" event={"ID":"34a1c006-cd05-48d7-9cb1-00635b1f4729","Type":"ContainerStarted","Data":"82b3736a0f51ffa837c8ce1432735316682b2938a4aee208ad615eacd85f2e3b"}
Oct 30 19:36:17.197643 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:17.197619    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4202bb04-nn2r6" event={"ID":"27c3c568-8107-46bf-8c4a-c35d7b549e4b","Type":"ContainerStarted","Data":"3ce5e9de314b49d8b242245d697b2b71d760d2a76a2929d4bfa8be1861dd1dd4"}
Oct 30 19:36:17.247767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:17.247705    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc" podStartSLOduration=1.247686571 podStartE2EDuration="1.247686571s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:36:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:36:17.242625761 +0000 UTC m=+889.085557701" watchObservedRunningTime="2026-10-30 19:36:17.247686571 +0000 UTC m=+889.090618515"
Oct 30 19:36:17.420887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:17.420395    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"
Oct 30 19:36:17.420887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:17.420582    5907 patch_prober.go:29] interesting pod/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused" start-of-body=
Oct 30 19:36:17.420887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:17.420660    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc" podUID="34a1c006-cd05-48d7-9cb1-00635b1f4729" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused"
Oct 30 19:36:17.503899 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:36:17.503847    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:36:18.207683 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:18.207559    5907 generic.go:356] "Generic (PLEG): container finished" podID="27c3c568-8107-46bf-8c4a-c35d7b549e4b" containerID="8a74ec580f59fd9f09d0d4acb07dae7f4a2ea2b4e45bc7de3cd05143486aefca" exitCode=0
Oct 30 19:36:18.207683 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:18.207649    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4202bb04-nn2r6" event={"ID":"27c3c568-8107-46bf-8c4a-c35d7b549e4b","Type":"ContainerDied","Data":"8a74ec580f59fd9f09d0d4acb07dae7f4a2ea2b4e45bc7de3cd05143486aefca"}
Oct 30 19:36:18.439885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:18.439181    5907 patch_prober.go:29] interesting pod/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused" start-of-body=
Oct 30 19:36:18.439885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:18.439263    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc" podUID="34a1c006-cd05-48d7-9cb1-00635b1f4729" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused"
Oct 30 19:36:19.434345 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:19.434306    5907 patch_prober.go:29] interesting pod/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused" start-of-body=
Oct 30 19:36:19.441735 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:19.441685    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc" podUID="34a1c006-cd05-48d7-9cb1-00635b1f4729" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused"
Oct 30 19:36:19.452437 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:19.452406    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4202bb04-nn2r6"
Oct 30 19:36:19.486194 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:19.486159    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/27c3c568-8107-46bf-8c4a-c35d7b549e4b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/27c3c568-8107-46bf-8c4a-c35d7b549e4b-maas-api-xks-ca\") pod \"27c3c568-8107-46bf-8c4a-c35d7b549e4b\" (UID: \"27c3c568-8107-46bf-8c4a-c35d7b549e4b\") "
Oct 30 19:36:19.486375 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:19.486315    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/27c3c568-8107-46bf-8c4a-c35d7b549e4b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/27c3c568-8107-46bf-8c4a-c35d7b549e4b-maas-api-openshift-ca\") pod \"27c3c568-8107-46bf-8c4a-c35d7b549e4b\" (UID: \"27c3c568-8107-46bf-8c4a-c35d7b549e4b\") "
Oct 30 19:36:19.486428 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:19.486409    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/27c3c568-8107-46bf-8c4a-c35d7b549e4b-kube-api-access-lzgvj\" (UniqueName: \"kubernetes.io/projected/27c3c568-8107-46bf-8c4a-c35d7b549e4b-kube-api-access-lzgvj\") pod \"27c3c568-8107-46bf-8c4a-c35d7b549e4b\" (UID: \"27c3c568-8107-46bf-8c4a-c35d7b549e4b\") "
Oct 30 19:36:19.486677 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:19.486649    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27c3c568-8107-46bf-8c4a-c35d7b549e4b-maas-api-openshift-ca" pod "27c3c568-8107-46bf-8c4a-c35d7b549e4b" (UID: "27c3c568-8107-46bf-8c4a-c35d7b549e4b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:36:19.486988 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:19.486972    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/27c3c568-8107-46bf-8c4a-c35d7b549e4b-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:19.488709 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:19.488681    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27c3c568-8107-46bf-8c4a-c35d7b549e4b-maas-api-xks-ca" pod "27c3c568-8107-46bf-8c4a-c35d7b549e4b" (UID: "27c3c568-8107-46bf-8c4a-c35d7b549e4b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:36:19.488806 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:19.488732    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27c3c568-8107-46bf-8c4a-c35d7b549e4b-kube-api-access-lzgvj" pod "27c3c568-8107-46bf-8c4a-c35d7b549e4b" (UID: "27c3c568-8107-46bf-8c4a-c35d7b549e4b"). InnerVolumeSpecName "kube-api-access-lzgvj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:36:19.588980 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:19.588938    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/27c3c568-8107-46bf-8c4a-c35d7b549e4b-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:19.588980 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:19.588977    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzgvj\" (UniqueName: \"kubernetes.io/projected/27c3c568-8107-46bf-8c4a-c35d7b549e4b-kube-api-access-lzgvj\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:20.257659 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:20.257623    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4202bb04-nn2r6" event={"ID":"27c3c568-8107-46bf-8c4a-c35d7b549e4b","Type":"ContainerDied","Data":"3ce5e9de314b49d8b242245d697b2b71d760d2a76a2929d4bfa8be1861dd1dd4"}
Oct 30 19:36:20.257850 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:20.257689    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3ce5e9de314b49d8b242245d697b2b71d760d2a76a2929d4bfa8be1861dd1dd4"
Oct 30 19:36:20.257850 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:20.257632    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4202bb04-nn2r6"
Oct 30 19:36:20.415660 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:20.415216    5907 patch_prober.go:29] interesting pod/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused" start-of-body=
Oct 30 19:36:20.415660 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:20.415278    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc" podUID="34a1c006-cd05-48d7-9cb1-00635b1f4729" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused"
Oct 30 19:36:21.306491 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:21.306449    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e308a20f-vskcl"]
Oct 30 19:36:21.308206 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:21.308150    5907 cpu_manager.go:414] "containerMap: removing container" podUID="27c3c568-8107-46bf-8c4a-c35d7b549e4b" containerName="revoke-keys"
Oct 30 19:36:21.308373 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:21.308356    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="27c3c568-8107-46bf-8c4a-c35d7b549e4b" containerName="revoke-keys"
Oct 30 19:36:21.308833 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:21.308813    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="27c3c568-8107-46bf-8c4a-c35d7b549e4b" containerName="revoke-keys"
Oct 30 19:36:21.347079 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:21.347039    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e308a20f-vskcl"
Oct 30 19:36:21.352052 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:21.352016    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e308a20f-vskcl"]
Oct 30 19:36:21.376576 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:21.376546    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:36:21.376742 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:21.376597    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:36:21.434882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:21.434205    5907 patch_prober.go:29] interesting pod/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused" start-of-body=
Oct 30 19:36:21.434882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:21.434271    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc" podUID="34a1c006-cd05-48d7-9cb1-00635b1f4729" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused"
Oct 30 19:36:21.452878 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:21.452832    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/987a4548-687f-46ee-bf92-0d9657ed2151-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e308a20f-vskcl\" (UID: \"987a4548-687f-46ee-bf92-0d9657ed2151\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e308a20f-vskcl"
Oct 30 19:36:21.453288 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:21.453271    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/987a4548-687f-46ee-bf92-0d9657ed2151-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e308a20f-vskcl\" (UID: \"987a4548-687f-46ee-bf92-0d9657ed2151\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e308a20f-vskcl"
Oct 30 19:36:21.453396 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:21.453383    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xfc4\" (UniqueName: \"kubernetes.io/projected/987a4548-687f-46ee-bf92-0d9657ed2151-kube-api-access-6xfc4\") pod \"maas-api-revoke-sub-e308a20f-vskcl\" (UID: \"987a4548-687f-46ee-bf92-0d9657ed2151\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e308a20f-vskcl"
Oct 30 19:36:21.555095 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:21.555044    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/987a4548-687f-46ee-bf92-0d9657ed2151-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e308a20f-vskcl\" (UID: \"987a4548-687f-46ee-bf92-0d9657ed2151\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e308a20f-vskcl"
Oct 30 19:36:21.555251 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:21.555135    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xfc4\" (UniqueName: \"kubernetes.io/projected/987a4548-687f-46ee-bf92-0d9657ed2151-kube-api-access-6xfc4\") pod \"maas-api-revoke-sub-e308a20f-vskcl\" (UID: \"987a4548-687f-46ee-bf92-0d9657ed2151\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e308a20f-vskcl"
Oct 30 19:36:21.555378 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:21.555357    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/987a4548-687f-46ee-bf92-0d9657ed2151-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e308a20f-vskcl\" (UID: \"987a4548-687f-46ee-bf92-0d9657ed2151\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e308a20f-vskcl"
Oct 30 19:36:21.556584 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:21.556490    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/987a4548-687f-46ee-bf92-0d9657ed2151-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e308a20f-vskcl\" (UID: \"987a4548-687f-46ee-bf92-0d9657ed2151\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e308a20f-vskcl"
Oct 30 19:36:21.557457 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:21.557432    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/987a4548-687f-46ee-bf92-0d9657ed2151-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e308a20f-vskcl\" (UID: \"987a4548-687f-46ee-bf92-0d9657ed2151\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e308a20f-vskcl"
Oct 30 19:36:21.577212 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:21.577186    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xfc4\" (UniqueName: \"kubernetes.io/projected/987a4548-687f-46ee-bf92-0d9657ed2151-kube-api-access-6xfc4\") pod \"maas-api-revoke-sub-e308a20f-vskcl\" (UID: \"987a4548-687f-46ee-bf92-0d9657ed2151\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e308a20f-vskcl"
Oct 30 19:36:21.588594 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:21.588566    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2495c2e8-xt2t2"]
Oct 30 19:36:21.594906 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:21.594885    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2495c2e8-xt2t2"
Oct 30 19:36:21.643477 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:21.643428    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2495c2e8-xt2t2"]
Oct 30 19:36:21.667457 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:21.667423    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e308a20f-vskcl"
Oct 30 19:36:21.777469 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:21.764901    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw9k4\" (UniqueName: \"kubernetes.io/projected/5fee739b-3a5d-4517-8b8a-ea26a4de9006-kube-api-access-nw9k4\") pod \"maas-api-revoke-sub-2495c2e8-xt2t2\" (UID: \"5fee739b-3a5d-4517-8b8a-ea26a4de9006\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2495c2e8-xt2t2"
Oct 30 19:36:21.777469 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:21.765025    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5fee739b-3a5d-4517-8b8a-ea26a4de9006-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2495c2e8-xt2t2\" (UID: \"5fee739b-3a5d-4517-8b8a-ea26a4de9006\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2495c2e8-xt2t2"
Oct 30 19:36:21.777469 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:21.765244    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5fee739b-3a5d-4517-8b8a-ea26a4de9006-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2495c2e8-xt2t2\" (UID: \"5fee739b-3a5d-4517-8b8a-ea26a4de9006\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2495c2e8-xt2t2"
Oct 30 19:36:21.899349 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:21.899315    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5fee739b-3a5d-4517-8b8a-ea26a4de9006-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2495c2e8-xt2t2\" (UID: \"5fee739b-3a5d-4517-8b8a-ea26a4de9006\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2495c2e8-xt2t2"
Oct 30 19:36:21.916971 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:21.909688    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5fee739b-3a5d-4517-8b8a-ea26a4de9006-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2495c2e8-xt2t2\" (UID: \"5fee739b-3a5d-4517-8b8a-ea26a4de9006\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2495c2e8-xt2t2"
Oct 30 19:36:21.935731 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:21.926952    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw9k4\" (UniqueName: \"kubernetes.io/projected/5fee739b-3a5d-4517-8b8a-ea26a4de9006-kube-api-access-nw9k4\") pod \"maas-api-revoke-sub-2495c2e8-xt2t2\" (UID: \"5fee739b-3a5d-4517-8b8a-ea26a4de9006\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2495c2e8-xt2t2"
Oct 30 19:36:21.972108 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:21.972073    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5fee739b-3a5d-4517-8b8a-ea26a4de9006-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2495c2e8-xt2t2\" (UID: \"5fee739b-3a5d-4517-8b8a-ea26a4de9006\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2495c2e8-xt2t2"
Oct 30 19:36:21.972607 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:21.972575    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw9k4\" (UniqueName: \"kubernetes.io/projected/5fee739b-3a5d-4517-8b8a-ea26a4de9006-kube-api-access-nw9k4\") pod \"maas-api-revoke-sub-2495c2e8-xt2t2\" (UID: \"5fee739b-3a5d-4517-8b8a-ea26a4de9006\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2495c2e8-xt2t2"
Oct 30 19:36:21.984886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:21.975107    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5fee739b-3a5d-4517-8b8a-ea26a4de9006-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2495c2e8-xt2t2\" (UID: \"5fee739b-3a5d-4517-8b8a-ea26a4de9006\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2495c2e8-xt2t2"
Oct 30 19:36:22.232940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:22.232652    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2495c2e8-xt2t2"
Oct 30 19:36:22.238762 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:22.238732    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e308a20f-vskcl"]
Oct 30 19:36:22.239527 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:36:22.239364    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod987a4548_687f_46ee_bf92_0d9657ed2151.slice/crio-1d51a32eef95bf73a5cf3f47a48c6c1d146cbd3951bcbc2c7ce5710551cc67a5 WatchSource:0}: Error finding container 1d51a32eef95bf73a5cf3f47a48c6c1d146cbd3951bcbc2c7ce5710551cc67a5: Status 404 returned error can't find the container with id 1d51a32eef95bf73a5cf3f47a48c6c1d146cbd3951bcbc2c7ce5710551cc67a5
Oct 30 19:36:22.277847 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:22.277805    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e308a20f-vskcl" event={"ID":"987a4548-687f-46ee-bf92-0d9657ed2151","Type":"ContainerStarted","Data":"1d51a32eef95bf73a5cf3f47a48c6c1d146cbd3951bcbc2c7ce5710551cc67a5"}
Oct 30 19:36:22.435041 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:22.435009    5907 patch_prober.go:29] interesting pod/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused" start-of-body=
Oct 30 19:36:22.435500 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:22.435478    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc" podUID="34a1c006-cd05-48d7-9cb1-00635b1f4729" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused"
Oct 30 19:36:22.567188 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:22.567151    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2495c2e8-xt2t2"]
Oct 30 19:36:23.288871 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:23.288831    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e308a20f-vskcl" event={"ID":"987a4548-687f-46ee-bf92-0d9657ed2151","Type":"ContainerStarted","Data":"ebf8ea7c835cad660ab905ef1cefae1808336d7b9404b10488b854243f7c8ac2"}
Oct 30 19:36:23.290631 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:23.290606    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2495c2e8-xt2t2" event={"ID":"5fee739b-3a5d-4517-8b8a-ea26a4de9006","Type":"ContainerStarted","Data":"8597a94fb771373d034e10b64022f3b5e5cac5e9e7184a779ee69d25a2d82310"}
Oct 30 19:36:23.290738 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:23.290648    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2495c2e8-xt2t2" event={"ID":"5fee739b-3a5d-4517-8b8a-ea26a4de9006","Type":"ContainerStarted","Data":"ae11c04773de24206f91f7f716728087065ae12187edb7c2b2d4fd18d91a8c69"}
Oct 30 19:36:23.338760 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:23.338712    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e308a20f-vskcl" podStartSLOduration=2.338699211 podStartE2EDuration="2.338699211s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:36:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:36:23.334410793 +0000 UTC m=+895.177342744" watchObservedRunningTime="2026-10-30 19:36:23.338699211 +0000 UTC m=+895.181631150"
Oct 30 19:36:23.389914 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:23.389852    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2495c2e8-xt2t2" podStartSLOduration=2.389837878 podStartE2EDuration="2.389837878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:36:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:36:23.383728676 +0000 UTC m=+895.226660615" watchObservedRunningTime="2026-10-30 19:36:23.389837878 +0000 UTC m=+895.232769865"
Oct 30 19:36:23.415228 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:23.415200    5907 patch_prober.go:29] interesting pod/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused" start-of-body=
Oct 30 19:36:23.415354 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:23.415248    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc" podUID="34a1c006-cd05-48d7-9cb1-00635b1f4729" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused"
Oct 30 19:36:24.299840 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:24.299804    5907 generic.go:356] "Generic (PLEG): container finished" podID="5fee739b-3a5d-4517-8b8a-ea26a4de9006" containerID="8597a94fb771373d034e10b64022f3b5e5cac5e9e7184a779ee69d25a2d82310" exitCode=0
Oct 30 19:36:24.300330 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:24.299959    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2495c2e8-xt2t2" event={"ID":"5fee739b-3a5d-4517-8b8a-ea26a4de9006","Type":"ContainerDied","Data":"8597a94fb771373d034e10b64022f3b5e5cac5e9e7184a779ee69d25a2d82310"}
Oct 30 19:36:24.302018 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:24.301991    5907 generic.go:356] "Generic (PLEG): container finished" podID="987a4548-687f-46ee-bf92-0d9657ed2151" containerID="ebf8ea7c835cad660ab905ef1cefae1808336d7b9404b10488b854243f7c8ac2" exitCode=0
Oct 30 19:36:24.302154 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:24.302077    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e308a20f-vskcl" event={"ID":"987a4548-687f-46ee-bf92-0d9657ed2151","Type":"ContainerDied","Data":"ebf8ea7c835cad660ab905ef1cefae1808336d7b9404b10488b854243f7c8ac2"}
Oct 30 19:36:24.428497 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:24.421379    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"
Oct 30 19:36:24.430513 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:24.430486    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"
Oct 30 19:36:24.431249 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:24.431212    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"
Oct 30 19:36:25.512543 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:25.512515    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e308a20f-vskcl"
Oct 30 19:36:25.517311 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:25.517288    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2495c2e8-xt2t2"
Oct 30 19:36:25.649909 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:25.649880    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/987a4548-687f-46ee-bf92-0d9657ed2151-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/987a4548-687f-46ee-bf92-0d9657ed2151-maas-api-openshift-ca\") pod \"987a4548-687f-46ee-bf92-0d9657ed2151\" (UID: \"987a4548-687f-46ee-bf92-0d9657ed2151\") "
Oct 30 19:36:25.650139 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:25.649930    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5fee739b-3a5d-4517-8b8a-ea26a4de9006-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5fee739b-3a5d-4517-8b8a-ea26a4de9006-maas-api-xks-ca\") pod \"5fee739b-3a5d-4517-8b8a-ea26a4de9006\" (UID: \"5fee739b-3a5d-4517-8b8a-ea26a4de9006\") "
Oct 30 19:36:25.650139 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:25.649984    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5fee739b-3a5d-4517-8b8a-ea26a4de9006-kube-api-access-nw9k4\" (UniqueName: \"kubernetes.io/projected/5fee739b-3a5d-4517-8b8a-ea26a4de9006-kube-api-access-nw9k4\") pod \"5fee739b-3a5d-4517-8b8a-ea26a4de9006\" (UID: \"5fee739b-3a5d-4517-8b8a-ea26a4de9006\") "
Oct 30 19:36:25.650139 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:25.650022    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5fee739b-3a5d-4517-8b8a-ea26a4de9006-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5fee739b-3a5d-4517-8b8a-ea26a4de9006-maas-api-openshift-ca\") pod \"5fee739b-3a5d-4517-8b8a-ea26a4de9006\" (UID: \"5fee739b-3a5d-4517-8b8a-ea26a4de9006\") "
Oct 30 19:36:25.650139 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:25.650058    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/987a4548-687f-46ee-bf92-0d9657ed2151-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/987a4548-687f-46ee-bf92-0d9657ed2151-maas-api-xks-ca\") pod \"987a4548-687f-46ee-bf92-0d9657ed2151\" (UID: \"987a4548-687f-46ee-bf92-0d9657ed2151\") "
Oct 30 19:36:25.650383 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:25.650221    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/987a4548-687f-46ee-bf92-0d9657ed2151-kube-api-access-6xfc4\" (UniqueName: \"kubernetes.io/projected/987a4548-687f-46ee-bf92-0d9657ed2151-kube-api-access-6xfc4\") pod \"987a4548-687f-46ee-bf92-0d9657ed2151\" (UID: \"987a4548-687f-46ee-bf92-0d9657ed2151\") "
Oct 30 19:36:25.650383 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:25.650324    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/987a4548-687f-46ee-bf92-0d9657ed2151-maas-api-openshift-ca" pod "987a4548-687f-46ee-bf92-0d9657ed2151" (UID: "987a4548-687f-46ee-bf92-0d9657ed2151"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:36:25.650498 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:25.650457    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fee739b-3a5d-4517-8b8a-ea26a4de9006-maas-api-openshift-ca" pod "5fee739b-3a5d-4517-8b8a-ea26a4de9006" (UID: "5fee739b-3a5d-4517-8b8a-ea26a4de9006"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:36:25.651152 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:25.651126    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/987a4548-687f-46ee-bf92-0d9657ed2151-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:25.651152 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:25.651153    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5fee739b-3a5d-4517-8b8a-ea26a4de9006-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:25.652496 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:25.652470    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fee739b-3a5d-4517-8b8a-ea26a4de9006-kube-api-access-nw9k4" pod "5fee739b-3a5d-4517-8b8a-ea26a4de9006" (UID: "5fee739b-3a5d-4517-8b8a-ea26a4de9006"). InnerVolumeSpecName "kube-api-access-nw9k4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:36:25.652601 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:25.652512    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fee739b-3a5d-4517-8b8a-ea26a4de9006-maas-api-xks-ca" pod "5fee739b-3a5d-4517-8b8a-ea26a4de9006" (UID: "5fee739b-3a5d-4517-8b8a-ea26a4de9006"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:36:25.652601 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:25.652528    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/987a4548-687f-46ee-bf92-0d9657ed2151-kube-api-access-6xfc4" pod "987a4548-687f-46ee-bf92-0d9657ed2151" (UID: "987a4548-687f-46ee-bf92-0d9657ed2151"). InnerVolumeSpecName "kube-api-access-6xfc4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:36:25.652768 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:25.652753    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/987a4548-687f-46ee-bf92-0d9657ed2151-maas-api-xks-ca" pod "987a4548-687f-46ee-bf92-0d9657ed2151" (UID: "987a4548-687f-46ee-bf92-0d9657ed2151"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:36:25.752561 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:25.752526    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xfc4\" (UniqueName: \"kubernetes.io/projected/987a4548-687f-46ee-bf92-0d9657ed2151-kube-api-access-6xfc4\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:25.752561 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:25.752566    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5fee739b-3a5d-4517-8b8a-ea26a4de9006-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:25.752766 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:25.752582    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nw9k4\" (UniqueName: \"kubernetes.io/projected/5fee739b-3a5d-4517-8b8a-ea26a4de9006-kube-api-access-nw9k4\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:25.752766 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:25.752596    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/987a4548-687f-46ee-bf92-0d9657ed2151-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:26.320211 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:26.320181    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2495c2e8-xt2t2"
Oct 30 19:36:26.320372 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:26.320213    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2495c2e8-xt2t2" event={"ID":"5fee739b-3a5d-4517-8b8a-ea26a4de9006","Type":"ContainerDied","Data":"ae11c04773de24206f91f7f716728087065ae12187edb7c2b2d4fd18d91a8c69"}
Oct 30 19:36:26.320372 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:26.320265    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae11c04773de24206f91f7f716728087065ae12187edb7c2b2d4fd18d91a8c69"
Oct 30 19:36:26.321830 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:26.321810    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e308a20f-vskcl"
Oct 30 19:36:26.321971 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:26.321893    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e308a20f-vskcl" event={"ID":"987a4548-687f-46ee-bf92-0d9657ed2151","Type":"ContainerDied","Data":"1d51a32eef95bf73a5cf3f47a48c6c1d146cbd3951bcbc2c7ce5710551cc67a5"}
Oct 30 19:36:26.321971 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:26.321930    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1d51a32eef95bf73a5cf3f47a48c6c1d146cbd3951bcbc2c7ce5710551cc67a5"
Oct 30 19:36:26.625428 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:26.625395    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1cf2851b-ss6cg"]
Oct 30 19:36:26.626177 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:26.626162    5907 cpu_manager.go:414] "containerMap: removing container" podUID="5fee739b-3a5d-4517-8b8a-ea26a4de9006" containerName="revoke-keys"
Oct 30 19:36:26.626230 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:26.626179    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5fee739b-3a5d-4517-8b8a-ea26a4de9006" containerName="revoke-keys"
Oct 30 19:36:26.626230 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:26.626198    5907 cpu_manager.go:414] "containerMap: removing container" podUID="987a4548-687f-46ee-bf92-0d9657ed2151" containerName="revoke-keys"
Oct 30 19:36:26.626230 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:26.626203    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="987a4548-687f-46ee-bf92-0d9657ed2151" containerName="revoke-keys"
Oct 30 19:36:26.626349 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:26.626339    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="5fee739b-3a5d-4517-8b8a-ea26a4de9006" containerName="revoke-keys"
Oct 30 19:36:26.626385 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:26.626355    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="987a4548-687f-46ee-bf92-0d9657ed2151" containerName="revoke-keys"
Oct 30 19:36:26.631306 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:26.631277    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1cf2851b-ss6cg"
Oct 30 19:36:26.650230 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:26.650173    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:36:26.650388 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:26.650266    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:36:26.656996 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:26.656965    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1cf2851b-ss6cg"]
Oct 30 19:36:26.811887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:26.776929    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a1e438ce-ee16-4984-8960-4a998d9515dc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1cf2851b-ss6cg\" (UID: \"a1e438ce-ee16-4984-8960-4a998d9515dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1cf2851b-ss6cg"
Oct 30 19:36:26.812282 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:26.812243    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqpjw\" (UniqueName: \"kubernetes.io/projected/a1e438ce-ee16-4984-8960-4a998d9515dc-kube-api-access-vqpjw\") pod \"maas-api-revoke-sub-1cf2851b-ss6cg\" (UID: \"a1e438ce-ee16-4984-8960-4a998d9515dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1cf2851b-ss6cg"
Oct 30 19:36:26.812835 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:26.812794    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a1e438ce-ee16-4984-8960-4a998d9515dc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1cf2851b-ss6cg\" (UID: \"a1e438ce-ee16-4984-8960-4a998d9515dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1cf2851b-ss6cg"
Oct 30 19:36:26.914649 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:26.914564    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a1e438ce-ee16-4984-8960-4a998d9515dc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1cf2851b-ss6cg\" (UID: \"a1e438ce-ee16-4984-8960-4a998d9515dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1cf2851b-ss6cg"
Oct 30 19:36:26.914824 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:26.914697    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a1e438ce-ee16-4984-8960-4a998d9515dc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1cf2851b-ss6cg\" (UID: \"a1e438ce-ee16-4984-8960-4a998d9515dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1cf2851b-ss6cg"
Oct 30 19:36:26.914824 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:26.914733    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqpjw\" (UniqueName: \"kubernetes.io/projected/a1e438ce-ee16-4984-8960-4a998d9515dc-kube-api-access-vqpjw\") pod \"maas-api-revoke-sub-1cf2851b-ss6cg\" (UID: \"a1e438ce-ee16-4984-8960-4a998d9515dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1cf2851b-ss6cg"
Oct 30 19:36:26.915345 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:26.915321    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a1e438ce-ee16-4984-8960-4a998d9515dc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1cf2851b-ss6cg\" (UID: \"a1e438ce-ee16-4984-8960-4a998d9515dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1cf2851b-ss6cg"
Oct 30 19:36:26.917445 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:26.917419    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a1e438ce-ee16-4984-8960-4a998d9515dc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1cf2851b-ss6cg\" (UID: \"a1e438ce-ee16-4984-8960-4a998d9515dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1cf2851b-ss6cg"
Oct 30 19:36:26.936487 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:26.936465    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqpjw\" (UniqueName: \"kubernetes.io/projected/a1e438ce-ee16-4984-8960-4a998d9515dc-kube-api-access-vqpjw\") pod \"maas-api-revoke-sub-1cf2851b-ss6cg\" (UID: \"a1e438ce-ee16-4984-8960-4a998d9515dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1cf2851b-ss6cg"
Oct 30 19:36:26.947318 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:26.947294    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1cf2851b-ss6cg"
Oct 30 19:36:27.258950 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:27.258922    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1cf2851b-ss6cg"]
Oct 30 19:36:27.335201 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:27.335166    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1cf2851b-ss6cg" event={"ID":"a1e438ce-ee16-4984-8960-4a998d9515dc","Type":"ContainerStarted","Data":"1387b7019d720685716b1964aa67ab720b71e69402a97b66f1858b0683de8bc7"}
Oct 30 19:36:27.554256 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:27.554220    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:36:28.514986 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:28.514321    5907 generic.go:356] "Generic (PLEG): container finished" podID="a1e438ce-ee16-4984-8960-4a998d9515dc" containerID="c0dc170c9b8029cd686672e625d5cb511772c7e67a4e838ea5e8e28f43f124f6" exitCode=0
Oct 30 19:36:28.514986 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:28.514375    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1cf2851b-ss6cg" event={"ID":"a1e438ce-ee16-4984-8960-4a998d9515dc","Type":"ContainerDied","Data":"c0dc170c9b8029cd686672e625d5cb511772c7e67a4e838ea5e8e28f43f124f6"}
Oct 30 19:36:28.523570 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:28.523537    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-44b833ff-9zk88"]
Oct 30 19:36:28.529110 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:28.529086    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-44b833ff-9zk88"
Oct 30 19:36:28.539555 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:28.539533    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-65fd42a4-8bv28"]
Oct 30 19:36:28.579059 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:28.579011    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-65fd42a4-8bv28"]
Oct 30 19:36:28.579248 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:28.579087    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-44b833ff-9zk88"]
Oct 30 19:36:28.579248 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:28.579230    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-65fd42a4-8bv28"
Oct 30 19:36:28.603132 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:28.599528    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vffxv\" (UniqueName: \"kubernetes.io/projected/d99e5ef4-84b7-4dd7-9251-b3aac9842f97-kube-api-access-vffxv\") pod \"maas-api-revoke-sub-44b833ff-9zk88\" (UID: \"d99e5ef4-84b7-4dd7-9251-b3aac9842f97\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-44b833ff-9zk88"
Oct 30 19:36:28.603132 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:28.599638    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d99e5ef4-84b7-4dd7-9251-b3aac9842f97-maas-api-xks-ca\") pod \"maas-api-revoke-sub-44b833ff-9zk88\" (UID: \"d99e5ef4-84b7-4dd7-9251-b3aac9842f97\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-44b833ff-9zk88"
Oct 30 19:36:28.603132 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:28.599701    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d99e5ef4-84b7-4dd7-9251-b3aac9842f97-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-44b833ff-9zk88\" (UID: \"d99e5ef4-84b7-4dd7-9251-b3aac9842f97\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-44b833ff-9zk88"
Oct 30 19:36:28.702409 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:28.702372    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c0228ac8-d5d3-4f74-8505-71d58292814a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-65fd42a4-8bv28\" (UID: \"c0228ac8-d5d3-4f74-8505-71d58292814a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-65fd42a4-8bv28"
Oct 30 19:36:28.702646 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:28.702629    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c0228ac8-d5d3-4f74-8505-71d58292814a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-65fd42a4-8bv28\" (UID: \"c0228ac8-d5d3-4f74-8505-71d58292814a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-65fd42a4-8bv28"
Oct 30 19:36:28.702965 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:28.702946    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vffxv\" (UniqueName: \"kubernetes.io/projected/d99e5ef4-84b7-4dd7-9251-b3aac9842f97-kube-api-access-vffxv\") pod \"maas-api-revoke-sub-44b833ff-9zk88\" (UID: \"d99e5ef4-84b7-4dd7-9251-b3aac9842f97\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-44b833ff-9zk88"
Oct 30 19:36:28.703164 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:28.703146    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d99e5ef4-84b7-4dd7-9251-b3aac9842f97-maas-api-xks-ca\") pod \"maas-api-revoke-sub-44b833ff-9zk88\" (UID: \"d99e5ef4-84b7-4dd7-9251-b3aac9842f97\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-44b833ff-9zk88"
Oct 30 19:36:28.703378 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:28.703314    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4csdt\" (UniqueName: \"kubernetes.io/projected/c0228ac8-d5d3-4f74-8505-71d58292814a-kube-api-access-4csdt\") pod \"maas-api-revoke-sub-65fd42a4-8bv28\" (UID: \"c0228ac8-d5d3-4f74-8505-71d58292814a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-65fd42a4-8bv28"
Oct 30 19:36:28.703723 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:28.703704    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d99e5ef4-84b7-4dd7-9251-b3aac9842f97-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-44b833ff-9zk88\" (UID: \"d99e5ef4-84b7-4dd7-9251-b3aac9842f97\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-44b833ff-9zk88"
Oct 30 19:36:28.707654 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:28.707598    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d99e5ef4-84b7-4dd7-9251-b3aac9842f97-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-44b833ff-9zk88\" (UID: \"d99e5ef4-84b7-4dd7-9251-b3aac9842f97\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-44b833ff-9zk88"
Oct 30 19:36:28.711665 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:28.711637    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d99e5ef4-84b7-4dd7-9251-b3aac9842f97-maas-api-xks-ca\") pod \"maas-api-revoke-sub-44b833ff-9zk88\" (UID: \"d99e5ef4-84b7-4dd7-9251-b3aac9842f97\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-44b833ff-9zk88"
Oct 30 19:36:28.743195 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:28.743119    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vffxv\" (UniqueName: \"kubernetes.io/projected/d99e5ef4-84b7-4dd7-9251-b3aac9842f97-kube-api-access-vffxv\") pod \"maas-api-revoke-sub-44b833ff-9zk88\" (UID: \"d99e5ef4-84b7-4dd7-9251-b3aac9842f97\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-44b833ff-9zk88"
Oct 30 19:36:28.806987 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:28.806847    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4csdt\" (UniqueName: \"kubernetes.io/projected/c0228ac8-d5d3-4f74-8505-71d58292814a-kube-api-access-4csdt\") pod \"maas-api-revoke-sub-65fd42a4-8bv28\" (UID: \"c0228ac8-d5d3-4f74-8505-71d58292814a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-65fd42a4-8bv28"
Oct 30 19:36:28.807368 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:28.807346    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c0228ac8-d5d3-4f74-8505-71d58292814a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-65fd42a4-8bv28\" (UID: \"c0228ac8-d5d3-4f74-8505-71d58292814a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-65fd42a4-8bv28"
Oct 30 19:36:28.807471 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:28.807458    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c0228ac8-d5d3-4f74-8505-71d58292814a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-65fd42a4-8bv28\" (UID: \"c0228ac8-d5d3-4f74-8505-71d58292814a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-65fd42a4-8bv28"
Oct 30 19:36:28.809849 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:28.808979    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c0228ac8-d5d3-4f74-8505-71d58292814a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-65fd42a4-8bv28\" (UID: \"c0228ac8-d5d3-4f74-8505-71d58292814a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-65fd42a4-8bv28"
Oct 30 19:36:28.841889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:28.841633    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c0228ac8-d5d3-4f74-8505-71d58292814a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-65fd42a4-8bv28\" (UID: \"c0228ac8-d5d3-4f74-8505-71d58292814a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-65fd42a4-8bv28"
Oct 30 19:36:28.841889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:28.841822    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4csdt\" (UniqueName: \"kubernetes.io/projected/c0228ac8-d5d3-4f74-8505-71d58292814a-kube-api-access-4csdt\") pod \"maas-api-revoke-sub-65fd42a4-8bv28\" (UID: \"c0228ac8-d5d3-4f74-8505-71d58292814a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-65fd42a4-8bv28"
Oct 30 19:36:28.881740 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:28.881441    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-44b833ff-9zk88"
Oct 30 19:36:28.948584 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:28.948537    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-65fd42a4-8bv28"
Oct 30 19:36:29.609652 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:29.609612    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7fbf748e-xx9cn"]
Oct 30 19:36:29.789886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:29.787098    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7fbf748e-xx9cn"]
Oct 30 19:36:29.789886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:29.787240    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7fbf748e-xx9cn"
Oct 30 19:36:30.054599 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:30.054556    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5493aa92-3882-424c-b4dd-ab462198db29-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7fbf748e-xx9cn\" (UID: \"5493aa92-3882-424c-b4dd-ab462198db29\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7fbf748e-xx9cn"
Oct 30 19:36:30.086890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:30.062067    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5493aa92-3882-424c-b4dd-ab462198db29-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7fbf748e-xx9cn\" (UID: \"5493aa92-3882-424c-b4dd-ab462198db29\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7fbf748e-xx9cn"
Oct 30 19:36:30.328381 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:30.122266    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mfwp\" (UniqueName: \"kubernetes.io/projected/5493aa92-3882-424c-b4dd-ab462198db29-kube-api-access-6mfwp\") pod \"maas-api-revoke-sub-7fbf748e-xx9cn\" (UID: \"5493aa92-3882-424c-b4dd-ab462198db29\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7fbf748e-xx9cn"
Oct 30 19:36:30.328381 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:30.212316    5907 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:30.328381 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:30.225609    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5493aa92-3882-424c-b4dd-ab462198db29-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7fbf748e-xx9cn\" (UID: \"5493aa92-3882-424c-b4dd-ab462198db29\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7fbf748e-xx9cn"
Oct 30 19:36:30.328381 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:30.225752    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5493aa92-3882-424c-b4dd-ab462198db29-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7fbf748e-xx9cn\" (UID: \"5493aa92-3882-424c-b4dd-ab462198db29\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7fbf748e-xx9cn"
Oct 30 19:36:30.328381 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:30.225851    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mfwp\" (UniqueName: \"kubernetes.io/projected/5493aa92-3882-424c-b4dd-ab462198db29-kube-api-access-6mfwp\") pod \"maas-api-revoke-sub-7fbf748e-xx9cn\" (UID: \"5493aa92-3882-424c-b4dd-ab462198db29\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7fbf748e-xx9cn"
Oct 30 19:36:30.328381 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:30.264260    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5493aa92-3882-424c-b4dd-ab462198db29-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7fbf748e-xx9cn\" (UID: \"5493aa92-3882-424c-b4dd-ab462198db29\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7fbf748e-xx9cn"
Oct 30 19:36:30.328381 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:30.264352    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:30.407162 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:30.398133    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:31.168948 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:31.168645    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mfwp\" (UniqueName: \"kubernetes.io/projected/5493aa92-3882-424c-b4dd-ab462198db29-kube-api-access-6mfwp\") pod \"maas-api-revoke-sub-7fbf748e-xx9cn\" (UID: \"5493aa92-3882-424c-b4dd-ab462198db29\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7fbf748e-xx9cn"
Oct 30 19:36:31.249640 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:31.249553    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5493aa92-3882-424c-b4dd-ab462198db29-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7fbf748e-xx9cn\" (UID: \"5493aa92-3882-424c-b4dd-ab462198db29\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7fbf748e-xx9cn"
Oct 30 19:36:31.303886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:31.299770    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 19:36:31.326050 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:31.326012    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:36:31.400905 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:31.400838    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:31.401456 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:31.401432    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:36:31.619084 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:31.618806    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7fbf748e-xx9cn"
Oct 30 19:36:31.647850 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:31.647793    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": context deadline exceeded"
Oct 30 19:36:31.817317 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:31.793633    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"]
Oct 30 19:36:32.038964 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.034097    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"]
Oct 30 19:36:32.065274 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.063996    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"
Oct 30 19:36:32.271889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.197786    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43db2eba-9244-4db6-ae67-8a20b01ea53e-istio-envoy\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"
Oct 30 19:36:32.271889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.197873    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43db2eba-9244-4db6-ae67-8a20b01ea53e-istio-podinfo\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"
Oct 30 19:36:32.271889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.197906    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43db2eba-9244-4db6-ae67-8a20b01ea53e-workload-certs\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"
Oct 30 19:36:32.271889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.197983    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43db2eba-9244-4db6-ae67-8a20b01ea53e-istiod-ca-cert\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"
Oct 30 19:36:32.271889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.198029    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43db2eba-9244-4db6-ae67-8a20b01ea53e-workload-socket\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"
Oct 30 19:36:32.271889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.198137    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vn6n\" (UniqueName: \"kubernetes.io/projected/43db2eba-9244-4db6-ae67-8a20b01ea53e-kube-api-access-6vn6n\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"
Oct 30 19:36:32.271889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.198207    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43db2eba-9244-4db6-ae67-8a20b01ea53e-istio-data\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"
Oct 30 19:36:32.271889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.198296    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43db2eba-9244-4db6-ae67-8a20b01ea53e-credential-socket\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"
Oct 30 19:36:32.271889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.198326    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43db2eba-9244-4db6-ae67-8a20b01ea53e-istio-token\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"
Oct 30 19:36:32.356891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.311390    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43db2eba-9244-4db6-ae67-8a20b01ea53e-istio-token\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"
Oct 30 19:36:32.378191 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.378157    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43db2eba-9244-4db6-ae67-8a20b01ea53e-istio-envoy\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"
Oct 30 19:36:32.378432 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.378413    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43db2eba-9244-4db6-ae67-8a20b01ea53e-istio-podinfo\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"
Oct 30 19:36:32.378565 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.378546    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43db2eba-9244-4db6-ae67-8a20b01ea53e-workload-certs\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"
Oct 30 19:36:32.378739 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.378724    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43db2eba-9244-4db6-ae67-8a20b01ea53e-istiod-ca-cert\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"
Oct 30 19:36:32.382885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.378881    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43db2eba-9244-4db6-ae67-8a20b01ea53e-workload-socket\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"
Oct 30 19:36:32.406921 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.383592    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43db2eba-9244-4db6-ae67-8a20b01ea53e-workload-socket\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"
Oct 30 19:36:32.407324 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.377448    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 19:36:32.432295 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.386686    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43db2eba-9244-4db6-ae67-8a20b01ea53e-workload-certs\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"
Oct 30 19:36:32.432578 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.387882    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43db2eba-9244-4db6-ae67-8a20b01ea53e-istiod-ca-cert\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"
Oct 30 19:36:32.439884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.407188    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:32.475046 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.407233    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 19:36:32.475046 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.447032    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"
Oct 30 19:36:32.550348 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.550246    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vn6n\" (UniqueName: \"kubernetes.io/projected/43db2eba-9244-4db6-ae67-8a20b01ea53e-kube-api-access-6vn6n\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"
Oct 30 19:36:32.594040 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.593999    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43db2eba-9244-4db6-ae67-8a20b01ea53e-istio-data\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"
Oct 30 19:36:32.673559 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.643271    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43db2eba-9244-4db6-ae67-8a20b01ea53e-credential-socket\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"
Oct 30 19:36:32.673559 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.644582    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43db2eba-9244-4db6-ae67-8a20b01ea53e-credential-socket\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"
Oct 30 19:36:32.673559 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.668676    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43db2eba-9244-4db6-ae67-8a20b01ea53e-istio-data\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"
Oct 30 19:36:32.727187 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.727142    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:32.793893 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.787816    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3504b2cf-8f1c-453e-9372-3d425261b29c-istio-data\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"
Oct 30 19:36:32.830190 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.812090    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3504b2cf-8f1c-453e-9372-3d425261b29c-istiod-ca-cert\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"
Oct 30 19:36:32.830190 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.812245    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3504b2cf-8f1c-453e-9372-3d425261b29c-istio-podinfo\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"
Oct 30 19:36:32.830190 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.812408    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3504b2cf-8f1c-453e-9372-3d425261b29c-workload-socket\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"
Oct 30 19:36:32.830190 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.812493    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3504b2cf-8f1c-453e-9372-3d425261b29c-credential-socket\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"
Oct 30 19:36:32.830190 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.812633    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pc4dj\" (UniqueName: \"kubernetes.io/projected/3504b2cf-8f1c-453e-9372-3d425261b29c-kube-api-access-pc4dj\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"
Oct 30 19:36:32.830190 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.812721    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3504b2cf-8f1c-453e-9372-3d425261b29c-istio-envoy\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"
Oct 30 19:36:32.830190 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.812824    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3504b2cf-8f1c-453e-9372-3d425261b29c-istio-token\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"
Oct 30 19:36:32.830190 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.817060    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3504b2cf-8f1c-453e-9372-3d425261b29c-workload-certs\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"
Oct 30 19:36:32.830190 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:32.817447    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43db2eba-9244-4db6-ae67-8a20b01ea53e-istio-podinfo\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"
Oct 30 19:36:33.154977 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:33.116565    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43db2eba-9244-4db6-ae67-8a20b01ea53e-istio-token\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"
Oct 30 19:36:33.154977 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:33.129115    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43db2eba-9244-4db6-ae67-8a20b01ea53e-istio-envoy\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"
Oct 30 19:36:33.310944 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:33.274735    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vn6n\" (UniqueName: \"kubernetes.io/projected/43db2eba-9244-4db6-ae67-8a20b01ea53e-kube-api-access-6vn6n\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"
Oct 30 19:36:33.310944 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:33.276626    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"
Oct 30 19:36:33.389785 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:33.342073    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3504b2cf-8f1c-453e-9372-3d425261b29c-istiod-ca-cert\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"
Oct 30 19:36:33.389785 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:33.342179    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3504b2cf-8f1c-453e-9372-3d425261b29c-istio-podinfo\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"
Oct 30 19:36:33.389785 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:33.342292    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3504b2cf-8f1c-453e-9372-3d425261b29c-workload-socket\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"
Oct 30 19:36:33.389785 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:33.342395    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3504b2cf-8f1c-453e-9372-3d425261b29c-credential-socket\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"
Oct 30 19:36:33.444125 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:33.443995    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:33.563930 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:33.457641    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pc4dj\" (UniqueName: \"kubernetes.io/projected/3504b2cf-8f1c-453e-9372-3d425261b29c-kube-api-access-pc4dj\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"
Oct 30 19:36:33.563930 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:33.549183    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3504b2cf-8f1c-453e-9372-3d425261b29c-istio-envoy\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"
Oct 30 19:36:33.563930 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:33.549400    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3504b2cf-8f1c-453e-9372-3d425261b29c-istio-token\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"
Oct 30 19:36:33.563930 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:33.549520    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3504b2cf-8f1c-453e-9372-3d425261b29c-workload-certs\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"
Oct 30 19:36:33.563930 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:33.549632    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3504b2cf-8f1c-453e-9372-3d425261b29c-istio-data\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"
Oct 30 19:36:33.606070 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:33.459083    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3504b2cf-8f1c-453e-9372-3d425261b29c-credential-socket\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"
Oct 30 19:36:33.685159 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:33.685113    5907 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:33.764891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:33.738195    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:34.133888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:34.110024    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3504b2cf-8f1c-453e-9372-3d425261b29c-workload-socket\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"
Oct 30 19:36:34.386558 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:33.459460    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:34.514846 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:33.513958    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3504b2cf-8f1c-453e-9372-3d425261b29c-istiod-ca-cert\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"
Oct 30 19:36:34.514846 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:34.452166    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3504b2cf-8f1c-453e-9372-3d425261b29c-workload-certs\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"
Oct 30 19:36:34.514846 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:34.509123    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3504b2cf-8f1c-453e-9372-3d425261b29c-istio-data\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"
Oct 30 19:36:34.543337 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:34.536192    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:34.817716 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:34.817657    5907 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wd6m6" podUID="0b52bf40-f75b-465b-8947-10ea73874f85" containerName="main" probeResult="failure" output="Get \"https://10.134.0.37:8000/health\": context deadline exceeded"
Oct 30 19:36:34.833590 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:34.818958    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:35.042505 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.042473    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3504b2cf-8f1c-453e-9372-3d425261b29c-istio-podinfo\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"
Oct 30 19:36:35.115036 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.101704    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3504b2cf-8f1c-453e-9372-3d425261b29c-istio-envoy\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"
Oct 30 19:36:35.139916 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.124941    5907 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-ingress/payload-processing-e2e-worker-w5-544fc8-78bc5bc8bd-wjdrd" podUID="615c6e69-747b-4294-ae5b-e0958527002c" containerName="payload-processing" probeResult="failure" output="dial tcp 10.134.0.50:50052: i/o timeout"
Oct 30 19:36:35.267087 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:36:35.158723    5907 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="2.716s"
Oct 30 19:36:35.267087 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.158792    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:36:35.267087 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.158818    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-44b833ff-9zk88" event={"ID":"d99e5ef4-84b7-4dd7-9251-b3aac9842f97","Type":"ContainerStarted","Data":"582957a66fd8219bfc6c74b48eac25d66f489a539cb4aa42f109df25d35d2a10"}
Oct 30 19:36:35.267087 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.158881    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"]
Oct 30 19:36:35.267087 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.158918    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"]
Oct 30 19:36:35.353958 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.307310    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": context deadline exceeded"
Oct 30 19:36:35.427788 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.427754    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3504b2cf-8f1c-453e-9372-3d425261b29c-istio-token\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"
Oct 30 19:36:35.460737 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.458886    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:35.473070 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.470340    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pc4dj\" (UniqueName: \"kubernetes.io/projected/3504b2cf-8f1c-453e-9372-3d425261b29c-kube-api-access-pc4dj\") pod \"e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") " pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"
Oct 30 19:36:35.505650 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.505565    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"]
Oct 30 19:36:35.518837 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.518800    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1cf2851b-ss6cg"
Oct 30 19:36:35.519820 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.519627    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-65fd42a4-8bv28" event={"ID":"c0228ac8-d5d3-4f74-8505-71d58292814a","Type":"ContainerStarted","Data":"f53fc16929b16c673ea4a0ae33090d0c8d410afdd1296578975920c0a5bac551"}
Oct 30 19:36:35.519820 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.519709    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:36:35.519820 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.519732    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"]
Oct 30 19:36:35.519820 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.519749    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"
Oct 30 19:36:35.519820 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.519765    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"]
Oct 30 19:36:35.524839 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.521427    5907 cpu_manager.go:414] "containerMap: removing container" podUID="a1e438ce-ee16-4984-8960-4a998d9515dc" containerName="revoke-keys"
Oct 30 19:36:35.524839 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.521450    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a1e438ce-ee16-4984-8960-4a998d9515dc" containerName="revoke-keys"
Oct 30 19:36:35.524839 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.521723    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="a1e438ce-ee16-4984-8960-4a998d9515dc" containerName="revoke-keys"
Oct 30 19:36:35.528908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.528887    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:36:35.529110 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.529098    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"]
Oct 30 19:36:35.529179 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.529171    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-65fd42a4-8bv28"]
Oct 30 19:36:35.529254 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.528936    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"
Oct 30 19:36:35.529547 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.529254    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:36:35.529659 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.529571    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:36:35.529659 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.529609    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:36:35.550350 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.550320    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"
Oct 30 19:36:35.566934 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.566065    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-44b833ff-9zk88"]
Oct 30 19:36:35.588142 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.588063    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7fbf748e-xx9cn"]
Oct 30 19:36:35.654460 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.653392    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a1e438ce-ee16-4984-8960-4a998d9515dc-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a1e438ce-ee16-4984-8960-4a998d9515dc-maas-api-xks-ca\") pod \"a1e438ce-ee16-4984-8960-4a998d9515dc\" (UID: \"a1e438ce-ee16-4984-8960-4a998d9515dc\") "
Oct 30 19:36:35.655003 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.654975    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a1e438ce-ee16-4984-8960-4a998d9515dc-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a1e438ce-ee16-4984-8960-4a998d9515dc-maas-api-openshift-ca\") pod \"a1e438ce-ee16-4984-8960-4a998d9515dc\" (UID: \"a1e438ce-ee16-4984-8960-4a998d9515dc\") "
Oct 30 19:36:35.655273 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.655246    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a1e438ce-ee16-4984-8960-4a998d9515dc-kube-api-access-vqpjw\" (UniqueName: \"kubernetes.io/projected/a1e438ce-ee16-4984-8960-4a998d9515dc-kube-api-access-vqpjw\") pod \"a1e438ce-ee16-4984-8960-4a998d9515dc\" (UID: \"a1e438ce-ee16-4984-8960-4a998d9515dc\") "
Oct 30 19:36:35.655887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.655841    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d44e37a-fe67-40e5-a3d4-c7665f508b76-workload-certs\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"
Oct 30 19:36:35.656045 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.656026    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f67f5efb-73b5-49fa-b044-36471405ce25-istio-envoy\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"
Oct 30 19:36:35.656206 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.656182    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d44e37a-fe67-40e5-a3d4-c7665f508b76-istio-envoy\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"
Oct 30 19:36:35.656363 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.656346    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d44e37a-fe67-40e5-a3d4-c7665f508b76-istio-podinfo\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"
Oct 30 19:36:35.656567 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.656549    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d44e37a-fe67-40e5-a3d4-c7665f508b76-istio-token\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"
Oct 30 19:36:35.657692 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.656735    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d44e37a-fe67-40e5-a3d4-c7665f508b76-credential-socket\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"
Oct 30 19:36:35.657899 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.657877    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f67f5efb-73b5-49fa-b044-36471405ce25-workload-socket\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"
Oct 30 19:36:35.658161 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.658142    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f67f5efb-73b5-49fa-b044-36471405ce25-istio-data\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"
Oct 30 19:36:35.658308 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.658287    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f67f5efb-73b5-49fa-b044-36471405ce25-workload-certs\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"
Oct 30 19:36:35.658481 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.658463    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d44e37a-fe67-40e5-a3d4-c7665f508b76-istio-data\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"
Oct 30 19:36:35.658678 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.658661    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d44e37a-fe67-40e5-a3d4-c7665f508b76-istiod-ca-cert\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"
Oct 30 19:36:35.659269 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.658808    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f67f5efb-73b5-49fa-b044-36471405ce25-istio-token\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"
Oct 30 19:36:35.659269 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.658948    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c54js\" (UniqueName: \"kubernetes.io/projected/5d44e37a-fe67-40e5-a3d4-c7665f508b76-kube-api-access-c54js\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"
Oct 30 19:36:35.659269 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.659071    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f67f5efb-73b5-49fa-b044-36471405ce25-istiod-ca-cert\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"
Oct 30 19:36:35.659269 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.659113    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d44e37a-fe67-40e5-a3d4-c7665f508b76-workload-socket\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"
Oct 30 19:36:35.659269 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.659150    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjdps\" (UniqueName: \"kubernetes.io/projected/f67f5efb-73b5-49fa-b044-36471405ce25-kube-api-access-fjdps\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"
Oct 30 19:36:35.659269 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.659206    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f67f5efb-73b5-49fa-b044-36471405ce25-istio-podinfo\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"
Oct 30 19:36:35.659269 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.659253    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f67f5efb-73b5-49fa-b044-36471405ce25-credential-socket\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"
Oct 30 19:36:35.664492 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.663284    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1e438ce-ee16-4984-8960-4a998d9515dc-maas-api-openshift-ca" pod "a1e438ce-ee16-4984-8960-4a998d9515dc" (UID: "a1e438ce-ee16-4984-8960-4a998d9515dc"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:36:35.674011 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.673984    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"]
Oct 30 19:36:35.712780 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.712614    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1e438ce-ee16-4984-8960-4a998d9515dc-maas-api-xks-ca" pod "a1e438ce-ee16-4984-8960-4a998d9515dc" (UID: "a1e438ce-ee16-4984-8960-4a998d9515dc"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:36:35.718906 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.713378    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1e438ce-ee16-4984-8960-4a998d9515dc-kube-api-access-vqpjw" pod "a1e438ce-ee16-4984-8960-4a998d9515dc" (UID: "a1e438ce-ee16-4984-8960-4a998d9515dc"). InnerVolumeSpecName "kube-api-access-vqpjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:36:35.722339 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.722292    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:36:35.722466 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.722385    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:36:35.722466 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.722435    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:36:35.761901 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.761769    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d44e37a-fe67-40e5-a3d4-c7665f508b76-istio-data\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"
Oct 30 19:36:35.763738 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.762587    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d44e37a-fe67-40e5-a3d4-c7665f508b76-istio-data\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"
Oct 30 19:36:35.763738 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.761848    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d44e37a-fe67-40e5-a3d4-c7665f508b76-istiod-ca-cert\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"
Oct 30 19:36:35.763738 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.762665    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f67f5efb-73b5-49fa-b044-36471405ce25-istio-token\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"
Oct 30 19:36:35.763738 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.762732    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c54js\" (UniqueName: \"kubernetes.io/projected/5d44e37a-fe67-40e5-a3d4-c7665f508b76-kube-api-access-c54js\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"
Oct 30 19:36:35.763738 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.762794    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f67f5efb-73b5-49fa-b044-36471405ce25-istiod-ca-cert\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"
Oct 30 19:36:35.763738 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.762820    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d44e37a-fe67-40e5-a3d4-c7665f508b76-workload-socket\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"
Oct 30 19:36:35.763738 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.762853    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjdps\" (UniqueName: \"kubernetes.io/projected/f67f5efb-73b5-49fa-b044-36471405ce25-kube-api-access-fjdps\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"
Oct 30 19:36:35.763738 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.762905    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f67f5efb-73b5-49fa-b044-36471405ce25-istio-podinfo\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"
Oct 30 19:36:35.763738 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.762946    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f67f5efb-73b5-49fa-b044-36471405ce25-credential-socket\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"
Oct 30 19:36:35.763738 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.763080    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d44e37a-fe67-40e5-a3d4-c7665f508b76-workload-certs\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"
Oct 30 19:36:35.764333 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.764213    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d44e37a-fe67-40e5-a3d4-c7665f508b76-workload-socket\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"
Oct 30 19:36:35.764807 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.764429    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f67f5efb-73b5-49fa-b044-36471405ce25-istio-envoy\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"
Oct 30 19:36:35.764807 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.764490    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d44e37a-fe67-40e5-a3d4-c7665f508b76-istio-envoy\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"
Oct 30 19:36:35.764807 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.764496    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f67f5efb-73b5-49fa-b044-36471405ce25-credential-socket\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"
Oct 30 19:36:35.764807 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.764548    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d44e37a-fe67-40e5-a3d4-c7665f508b76-istio-podinfo\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"
Oct 30 19:36:35.764807 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.764590    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d44e37a-fe67-40e5-a3d4-c7665f508b76-istio-token\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"
Oct 30 19:36:35.764807 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.764601    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f67f5efb-73b5-49fa-b044-36471405ce25-istiod-ca-cert\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"
Oct 30 19:36:35.764807 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.764633    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d44e37a-fe67-40e5-a3d4-c7665f508b76-credential-socket\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"
Oct 30 19:36:35.764807 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.764688    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f67f5efb-73b5-49fa-b044-36471405ce25-workload-socket\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"
Oct 30 19:36:35.764807 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.764787    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f67f5efb-73b5-49fa-b044-36471405ce25-istio-data\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"
Oct 30 19:36:35.765352 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.764829    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f67f5efb-73b5-49fa-b044-36471405ce25-workload-certs\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"
Oct 30 19:36:35.765352 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.765010    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a1e438ce-ee16-4984-8960-4a998d9515dc-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:35.765352 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.765029    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vqpjw\" (UniqueName: \"kubernetes.io/projected/a1e438ce-ee16-4984-8960-4a998d9515dc-kube-api-access-vqpjw\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:35.765352 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.765038    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d44e37a-fe67-40e5-a3d4-c7665f508b76-istiod-ca-cert\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"
Oct 30 19:36:35.765352 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.765046    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a1e438ce-ee16-4984-8960-4a998d9515dc-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:35.765352 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.765300    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f67f5efb-73b5-49fa-b044-36471405ce25-workload-certs\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"
Oct 30 19:36:35.765352 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.765321    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d44e37a-fe67-40e5-a3d4-c7665f508b76-credential-socket\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"
Oct 30 19:36:35.765725 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.765493    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d44e37a-fe67-40e5-a3d4-c7665f508b76-workload-certs\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"
Oct 30 19:36:35.765725 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.765546    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f67f5efb-73b5-49fa-b044-36471405ce25-workload-socket\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"
Oct 30 19:36:35.765834 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.765809    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f67f5efb-73b5-49fa-b044-36471405ce25-istio-data\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"
Oct 30 19:36:35.767348 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.767328    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f67f5efb-73b5-49fa-b044-36471405ce25-istio-podinfo\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"
Oct 30 19:36:35.768724 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.768703    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d44e37a-fe67-40e5-a3d4-c7665f508b76-istio-podinfo\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"
Oct 30 19:36:35.769446 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.769397    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d44e37a-fe67-40e5-a3d4-c7665f508b76-istio-envoy\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"
Oct 30 19:36:35.769601 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.769564    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f67f5efb-73b5-49fa-b044-36471405ce25-istio-envoy\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"
Oct 30 19:36:35.789168 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.788806    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d44e37a-fe67-40e5-a3d4-c7665f508b76-istio-token\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"
Oct 30 19:36:35.789943 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.789908    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f67f5efb-73b5-49fa-b044-36471405ce25-istio-token\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"
Oct 30 19:36:35.791644 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.791607    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjdps\" (UniqueName: \"kubernetes.io/projected/f67f5efb-73b5-49fa-b044-36471405ce25-kube-api-access-fjdps\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"
Oct 30 19:36:35.793134 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.793110    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c54js\" (UniqueName: \"kubernetes.io/projected/5d44e37a-fe67-40e5-a3d4-c7665f508b76-kube-api-access-c54js\") pod \"e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") " pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"
Oct 30 19:36:35.915209 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.915163    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:36:35.915339 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.915310    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:36:35.915441 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.915413    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:36:35.918455 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.918424    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"]
Oct 30 19:36:35.951516 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.951485    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"
Oct 30 19:36:35.952354 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:35.952127    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"
Oct 30 19:36:36.296802 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:36.296769    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"]
Oct 30 19:36:36.319899 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:36.319824    5907 generic.go:356] "Generic (PLEG): container finished" podID="c0228ac8-d5d3-4f74-8505-71d58292814a" containerID="312a31e39c0f98a2aaef81c13587743a31013e69ccc3080e9a6e3a00f5b41edc" exitCode=0
Oct 30 19:36:36.320048 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:36.319893    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-65fd42a4-8bv28" event={"ID":"c0228ac8-d5d3-4f74-8505-71d58292814a","Type":"ContainerDied","Data":"312a31e39c0f98a2aaef81c13587743a31013e69ccc3080e9a6e3a00f5b41edc"}
Oct 30 19:36:36.323707 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:36.322650    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn" event={"ID":"43db2eba-9244-4db6-ae67-8a20b01ea53e","Type":"ContainerStarted","Data":"161dbeed1b1a6cfe0da1d965482bb0878c9ed0c873dfdf879651affd2560f856"}
Oct 30 19:36:36.323707 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:36.322709    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn" event={"ID":"43db2eba-9244-4db6-ae67-8a20b01ea53e","Type":"ContainerStarted","Data":"866028d1062316c1d10d3396e0c0c8c7cc877c3775baae13fe8f5ac1517f5c16"}
Oct 30 19:36:36.326348 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:36.326322    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8" event={"ID":"3504b2cf-8f1c-453e-9372-3d425261b29c","Type":"ContainerStarted","Data":"8794bae25ae02f8e1d05acd7c30647daed122dbc6bb4cbfe0543a5724d29aa38"}
Oct 30 19:36:36.326508 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:36.326373    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8" event={"ID":"3504b2cf-8f1c-453e-9372-3d425261b29c","Type":"ContainerStarted","Data":"dfa5e0ee1d238417fbc064aeabc0bd472aac2eb554a712e508adb9a752d1c1af"}
Oct 30 19:36:36.332369 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:36.331079    5907 generic.go:356] "Generic (PLEG): container finished" podID="d99e5ef4-84b7-4dd7-9251-b3aac9842f97" containerID="175b0ac83636e672c808236b06efa38eb3d1bcee8060a8cec2a62ac3dbbba9be" exitCode=0
Oct 30 19:36:36.332369 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:36.331143    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-44b833ff-9zk88" event={"ID":"d99e5ef4-84b7-4dd7-9251-b3aac9842f97","Type":"ContainerDied","Data":"175b0ac83636e672c808236b06efa38eb3d1bcee8060a8cec2a62ac3dbbba9be"}
Oct 30 19:36:36.332369 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:36.332295    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"]
Oct 30 19:36:36.334854 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:36.334830    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7fbf748e-xx9cn" event={"ID":"5493aa92-3882-424c-b4dd-ab462198db29","Type":"ContainerStarted","Data":"8773732a20df86c8808c02fc130493ca9287771283707c1351381cb6aaeab47e"}
Oct 30 19:36:36.334991 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:36.334906    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7fbf748e-xx9cn" event={"ID":"5493aa92-3882-424c-b4dd-ab462198db29","Type":"ContainerStarted","Data":"9a6e11c8fa2d3eefd24ba96e6125dbf9c0532bd2fa442a951c71e1022c6de03b"}
Oct 30 19:36:36.337336 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:36.337304    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:36:36.337466 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:36.337384    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:36:36.337466 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:36.337434    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:36:36.342088 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:36.342068    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1cf2851b-ss6cg"
Oct 30 19:36:36.342220 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:36.342087    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1cf2851b-ss6cg" event={"ID":"a1e438ce-ee16-4984-8960-4a998d9515dc","Type":"ContainerDied","Data":"1387b7019d720685716b1964aa67ab720b71e69402a97b66f1858b0683de8bc7"}
Oct 30 19:36:36.342307 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:36.342250    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1387b7019d720685716b1964aa67ab720b71e69402a97b66f1858b0683de8bc7"
Oct 30 19:36:36.343962 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:36.343758    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:36:36.343962 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:36.343832    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:36:36.343962 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:36.343925    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:36:36.423231 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:36.420371    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn" podStartSLOduration=6.420349049 podStartE2EDuration="6.420349049s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:36:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:36:36.417235685 +0000 UTC m=+908.260167626" watchObservedRunningTime="2026-10-30 19:36:36.420349049 +0000 UTC m=+908.263280988"
Oct 30 19:36:36.496219 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:36.496127    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8" podStartSLOduration=6.496103262 podStartE2EDuration="6.496103262s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:36:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:36:36.477988751 +0000 UTC m=+908.320920690" watchObservedRunningTime="2026-10-30 19:36:36.496103262 +0000 UTC m=+908.339035203"
Oct 30 19:36:36.523548 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:36.519357    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"
Oct 30 19:36:36.535998 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:36.529401    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Oct 30 19:36:36.535998 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:36.529474    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn" podUID="43db2eba-9244-4db6-ae67-8a20b01ea53e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Oct 30 19:36:36.558295 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:36.558215    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7fbf748e-xx9cn" podStartSLOduration=7.5581913 podStartE2EDuration="7.5581913s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:36:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:36:36.547511931 +0000 UTC m=+908.390443876" watchObservedRunningTime="2026-10-30 19:36:36.5581913 +0000 UTC m=+908.401123234"
Oct 30 19:36:36.561134 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:36.560542    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"
Oct 30 19:36:37.389282 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:37.389230    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2" event={"ID":"5d44e37a-fe67-40e5-a3d4-c7665f508b76","Type":"ContainerStarted","Data":"ec97339621ffc11976e39b69fcc5b9876e0d56428d6a2f6ace4f447a1f41a41a"}
Oct 30 19:36:37.389567 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:37.389533    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2" event={"ID":"5d44e37a-fe67-40e5-a3d4-c7665f508b76","Type":"ContainerStarted","Data":"8327b093069ff2b70251d3d40b8b8faf4cf1d3d6750c7431e3297f5533e4bc87"}
Oct 30 19:36:37.393215 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:37.393188    5907 generic.go:356] "Generic (PLEG): container finished" podID="5493aa92-3882-424c-b4dd-ab462198db29" containerID="8773732a20df86c8808c02fc130493ca9287771283707c1351381cb6aaeab47e" exitCode=0
Oct 30 19:36:37.393361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:37.393307    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7fbf748e-xx9cn" event={"ID":"5493aa92-3882-424c-b4dd-ab462198db29","Type":"ContainerDied","Data":"8773732a20df86c8808c02fc130493ca9287771283707c1351381cb6aaeab47e"}
Oct 30 19:36:37.396564 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:37.396504    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc" event={"ID":"f67f5efb-73b5-49fa-b044-36471405ce25","Type":"ContainerStarted","Data":"fb40a12db00274f0fc33da971134b8ce534acaed2fdd623eef70c8179ab054b3"}
Oct 30 19:36:37.399632 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:37.399575    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc" event={"ID":"f67f5efb-73b5-49fa-b044-36471405ce25","Type":"ContainerStarted","Data":"caea4fd122a4dd6aca263d0daa30215d103d8fe1ac9f5c6c64db40f7f8ad2667"}
Oct 30 19:36:37.510958 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:37.510450    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Oct 30 19:36:37.510958 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:37.510508    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn" podUID="43db2eba-9244-4db6-ae67-8a20b01ea53e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Oct 30 19:36:37.566750 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:37.566685    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2" podStartSLOduration=7.566665081 podStartE2EDuration="7.566665081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:36:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:36:37.510976468 +0000 UTC m=+909.353908412" watchObservedRunningTime="2026-10-30 19:36:37.566665081 +0000 UTC m=+909.409597022"
Oct 30 19:36:37.590563 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:37.590528    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused" start-of-body=
Oct 30 19:36:37.593653 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:37.593619    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8" podUID="3504b2cf-8f1c-453e-9372-3d425261b29c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused"
Oct 30 19:36:37.668895 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:37.667232    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc" podStartSLOduration=7.667209306 podStartE2EDuration="7.667209306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:36:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:36:37.633338552 +0000 UTC m=+909.476270492" watchObservedRunningTime="2026-10-30 19:36:37.667209306 +0000 UTC m=+909.510141245"
Oct 30 19:36:37.978897 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:37.978512    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"
Oct 30 19:36:38.008359 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:38.007077    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"
Oct 30 19:36:38.029946 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:38.029015    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Oct 30 19:36:38.029946 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:38.029082    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc" podUID="f67f5efb-73b5-49fa-b044-36471405ce25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Oct 30 19:36:38.031713 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:38.031599    5907 patch_prober.go:29] interesting pod/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 30 19:36:38.031713 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:38.031655    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2" podUID="5d44e37a-fe67-40e5-a3d4-c7665f508b76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 30 19:36:38.499407 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:38.498195    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 19:36:38.499407 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:38.498714    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:38.499407 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:38.499332    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": context deadline exceeded"
Oct 30 19:36:38.596918 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:38.591964    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:38.925887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:38.902840    5907 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:39.678882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:39.678625    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Oct 30 19:36:39.678882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:39.678754    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn" podUID="43db2eba-9244-4db6-ae67-8a20b01ea53e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Oct 30 19:36:39.700890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:39.680227    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Oct 30 19:36:39.701150 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:39.680389    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn" podUID="43db2eba-9244-4db6-ae67-8a20b01ea53e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Oct 30 19:36:39.751076 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:39.732382    5907 patch_prober.go:29] interesting pod/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 30 19:36:39.751076 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:39.732448    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2" podUID="5d44e37a-fe67-40e5-a3d4-c7665f508b76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 30 19:36:39.836879 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:39.766492    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused" start-of-body=
Oct 30 19:36:39.836879 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:39.766560    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8" podUID="3504b2cf-8f1c-453e-9372-3d425261b29c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused"
Oct 30 19:36:39.836879 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:39.785107    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused" start-of-body=
Oct 30 19:36:39.836879 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:39.785176    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8" podUID="3504b2cf-8f1c-453e-9372-3d425261b29c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused"
Oct 30 19:36:39.836879 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:36:39.791541    5907 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.442s"
Oct 30 19:36:40.048369 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:40.014756    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Oct 30 19:36:40.048369 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:40.014842    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc" podUID="f67f5efb-73b5-49fa-b044-36471405ce25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Oct 30 19:36:40.628061 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:40.628017    5907 patch_prober.go:29] interesting pod/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 30 19:36:40.628267 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:40.628090    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2" podUID="5d44e37a-fe67-40e5-a3d4-c7665f508b76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 30 19:36:40.677889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:40.674459    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Oct 30 19:36:40.738907 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:40.730292    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn" podUID="43db2eba-9244-4db6-ae67-8a20b01ea53e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Oct 30 19:36:40.738907 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:40.716942    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:40.738907 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:40.733544    5907 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:40.893998 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:40.881998    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:40.893998 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:40.883820    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 19:36:40.893998 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:40.679171    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused" start-of-body=
Oct 30 19:36:40.893998 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:40.886602    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8" podUID="3504b2cf-8f1c-453e-9372-3d425261b29c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused"
Oct 30 19:36:40.893998 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:40.692216    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Oct 30 19:36:40.893998 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:40.886648    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc" podUID="f67f5efb-73b5-49fa-b044-36471405ce25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Oct 30 19:36:41.034140 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.034105    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Oct 30 19:36:41.034282 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.034166    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc" podUID="f67f5efb-73b5-49fa-b044-36471405ce25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Oct 30 19:36:41.061192 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.061145    5907 patch_prober.go:29] interesting pod/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 30 19:36:41.061564 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.061509    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2" podUID="5d44e37a-fe67-40e5-a3d4-c7665f508b76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 30 19:36:41.243946 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.242311    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-65fd42a4-8bv28"
Oct 30 19:36:41.313894 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.296079    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c0228ac8-d5d3-4f74-8505-71d58292814a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c0228ac8-d5d3-4f74-8505-71d58292814a-maas-api-xks-ca\") pod \"c0228ac8-d5d3-4f74-8505-71d58292814a\" (UID: \"c0228ac8-d5d3-4f74-8505-71d58292814a\") "
Oct 30 19:36:41.313894 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.296138    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c0228ac8-d5d3-4f74-8505-71d58292814a-kube-api-access-4csdt\" (UniqueName: \"kubernetes.io/projected/c0228ac8-d5d3-4f74-8505-71d58292814a-kube-api-access-4csdt\") pod \"c0228ac8-d5d3-4f74-8505-71d58292814a\" (UID: \"c0228ac8-d5d3-4f74-8505-71d58292814a\") "
Oct 30 19:36:41.313894 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.296207    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c0228ac8-d5d3-4f74-8505-71d58292814a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c0228ac8-d5d3-4f74-8505-71d58292814a-maas-api-openshift-ca\") pod \"c0228ac8-d5d3-4f74-8505-71d58292814a\" (UID: \"c0228ac8-d5d3-4f74-8505-71d58292814a\") "
Oct 30 19:36:41.321023 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.320972    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:41.368026 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.357307    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0228ac8-d5d3-4f74-8505-71d58292814a-maas-api-openshift-ca" pod "c0228ac8-d5d3-4f74-8505-71d58292814a" (UID: "c0228ac8-d5d3-4f74-8505-71d58292814a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:36:41.377944 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.369111    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-44b833ff-9zk88"
Oct 30 19:36:41.409886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.405489    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c0228ac8-d5d3-4f74-8505-71d58292814a-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:41.426102 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.426062    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0228ac8-d5d3-4f74-8505-71d58292814a-maas-api-xks-ca" pod "c0228ac8-d5d3-4f74-8505-71d58292814a" (UID: "c0228ac8-d5d3-4f74-8505-71d58292814a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:36:41.427959 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.427075    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0228ac8-d5d3-4f74-8505-71d58292814a-kube-api-access-4csdt" pod "c0228ac8-d5d3-4f74-8505-71d58292814a" (UID: "c0228ac8-d5d3-4f74-8505-71d58292814a"). InnerVolumeSpecName "kube-api-access-4csdt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:36:41.509759 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.509661    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Oct 30 19:36:41.510060 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.510026    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn" podUID="43db2eba-9244-4db6-ae67-8a20b01ea53e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Oct 30 19:36:41.527349 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.527317    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d99e5ef4-84b7-4dd7-9251-b3aac9842f97-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d99e5ef4-84b7-4dd7-9251-b3aac9842f97-maas-api-openshift-ca\") pod \"d99e5ef4-84b7-4dd7-9251-b3aac9842f97\" (UID: \"d99e5ef4-84b7-4dd7-9251-b3aac9842f97\") "
Oct 30 19:36:41.528026 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.527997    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d99e5ef4-84b7-4dd7-9251-b3aac9842f97-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d99e5ef4-84b7-4dd7-9251-b3aac9842f97-maas-api-xks-ca\") pod \"d99e5ef4-84b7-4dd7-9251-b3aac9842f97\" (UID: \"d99e5ef4-84b7-4dd7-9251-b3aac9842f97\") "
Oct 30 19:36:41.589918 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.587627    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d99e5ef4-84b7-4dd7-9251-b3aac9842f97-kube-api-access-vffxv\" (UniqueName: \"kubernetes.io/projected/d99e5ef4-84b7-4dd7-9251-b3aac9842f97-kube-api-access-vffxv\") pod \"d99e5ef4-84b7-4dd7-9251-b3aac9842f97\" (UID: \"d99e5ef4-84b7-4dd7-9251-b3aac9842f97\") "
Oct 30 19:36:41.589918 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.588554    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c0228ac8-d5d3-4f74-8505-71d58292814a-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:41.589918 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.588583    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4csdt\" (UniqueName: \"kubernetes.io/projected/c0228ac8-d5d3-4f74-8505-71d58292814a-kube-api-access-4csdt\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:41.653254 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.610517    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused" start-of-body=
Oct 30 19:36:41.653254 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.610581    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8" podUID="3504b2cf-8f1c-453e-9372-3d425261b29c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused"
Oct 30 19:36:41.653254 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.561257    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:36:41.653254 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.611126    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-44b833ff-9zk88" event={"ID":"d99e5ef4-84b7-4dd7-9251-b3aac9842f97","Type":"ContainerDied","Data":"582957a66fd8219bfc6c74b48eac25d66f489a539cb4aa42f109df25d35d2a10"}
Oct 30 19:36:41.653254 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.611173    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="582957a66fd8219bfc6c74b48eac25d66f489a539cb4aa42f109df25d35d2a10"
Oct 30 19:36:41.653254 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.611326    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:36:41.653254 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.611371    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:36:41.653254 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.621235    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d99e5ef4-84b7-4dd7-9251-b3aac9842f97-maas-api-openshift-ca" pod "d99e5ef4-84b7-4dd7-9251-b3aac9842f97" (UID: "d99e5ef4-84b7-4dd7-9251-b3aac9842f97"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:36:41.653254 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.651010    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-65fd42a4-8bv28"
Oct 30 19:36:41.653254 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.651127    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-65fd42a4-8bv28" event={"ID":"c0228ac8-d5d3-4f74-8505-71d58292814a","Type":"ContainerDied","Data":"f53fc16929b16c673ea4a0ae33090d0c8d410afdd1296578975920c0a5bac551"}
Oct 30 19:36:41.653254 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.651169    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f53fc16929b16c673ea4a0ae33090d0c8d410afdd1296578975920c0a5bac551"
Oct 30 19:36:41.672107 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.670071    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d99e5ef4-84b7-4dd7-9251-b3aac9842f97-kube-api-access-vffxv" pod "d99e5ef4-84b7-4dd7-9251-b3aac9842f97" (UID: "d99e5ef4-84b7-4dd7-9251-b3aac9842f97"). InnerVolumeSpecName "kube-api-access-vffxv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:36:41.750361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.750319    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:41.812929 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.812894    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d99e5ef4-84b7-4dd7-9251-b3aac9842f97-maas-api-xks-ca" pod "d99e5ef4-84b7-4dd7-9251-b3aac9842f97" (UID: "d99e5ef4-84b7-4dd7-9251-b3aac9842f97"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:36:41.957889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.950243    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:41.957889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.950666    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": context deadline exceeded"
Oct 30 19:36:41.989883 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.987431    5907 patch_prober.go:29] interesting pod/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 30 19:36:41.989883 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.988433    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Oct 30 19:36:41.989883 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.988490    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc" podUID="f67f5efb-73b5-49fa-b044-36471405ce25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Oct 30 19:36:41.991007 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:41.987491    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2" podUID="5d44e37a-fe67-40e5-a3d4-c7665f508b76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 30 19:36:42.005920 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:42.000637    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d99e5ef4-84b7-4dd7-9251-b3aac9842f97-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d99e5ef4-84b7-4dd7-9251-b3aac9842f97-maas-api-xks-ca\") pod \"d99e5ef4-84b7-4dd7-9251-b3aac9842f97\" (UID: \"d99e5ef4-84b7-4dd7-9251-b3aac9842f97\") "
Oct 30 19:36:42.005920 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:36:42.001161    5907 empty_dir.go:505] Warning: Unmount skipped because path does not exist: /var/lib/kubelet/pods/d99e5ef4-84b7-4dd7-9251-b3aac9842f97/volumes/kubernetes.io~secret/maas-api-xks-ca
Oct 30 19:36:42.005920 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:42.001184    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d99e5ef4-84b7-4dd7-9251-b3aac9842f97-maas-api-xks-ca" pod "d99e5ef4-84b7-4dd7-9251-b3aac9842f97" (UID: "d99e5ef4-84b7-4dd7-9251-b3aac9842f97"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:36:42.209959 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:42.095264    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d99e5ef4-84b7-4dd7-9251-b3aac9842f97-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:42.271208 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:42.271154    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vffxv\" (UniqueName: \"kubernetes.io/projected/d99e5ef4-84b7-4dd7-9251-b3aac9842f97-kube-api-access-vffxv\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:42.271472 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:42.271450    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d99e5ef4-84b7-4dd7-9251-b3aac9842f97-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:42.449883 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:42.417926    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": context deadline exceeded"
Oct 30 19:36:42.570464 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:42.570372    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused" start-of-body=
Oct 30 19:36:42.570700 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:42.570671    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Oct 30 19:36:42.570780 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:42.570723    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn" podUID="43db2eba-9244-4db6-ae67-8a20b01ea53e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Oct 30 19:36:42.570900 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:42.570688    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8" podUID="3504b2cf-8f1c-453e-9372-3d425261b29c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused"
Oct 30 19:36:42.638200 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:42.638158    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7fbf748e-xx9cn"
Oct 30 19:36:42.715887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:42.712286    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:36:42.892004 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:42.891976    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:36:42.914038 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:42.914003    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:36:42.914305 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:42.798386    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5493aa92-3882-424c-b4dd-ab462198db29-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5493aa92-3882-424c-b4dd-ab462198db29-maas-api-xks-ca\") pod \"5493aa92-3882-424c-b4dd-ab462198db29\" (UID: \"5493aa92-3882-424c-b4dd-ab462198db29\") "
Oct 30 19:36:42.914762 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:42.914737    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5493aa92-3882-424c-b4dd-ab462198db29-kube-api-access-6mfwp\" (UniqueName: \"kubernetes.io/projected/5493aa92-3882-424c-b4dd-ab462198db29-kube-api-access-6mfwp\") pod \"5493aa92-3882-424c-b4dd-ab462198db29\" (UID: \"5493aa92-3882-424c-b4dd-ab462198db29\") "
Oct 30 19:36:42.927844 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:42.927814    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5493aa92-3882-424c-b4dd-ab462198db29-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5493aa92-3882-424c-b4dd-ab462198db29-maas-api-openshift-ca\") pod \"5493aa92-3882-424c-b4dd-ab462198db29\" (UID: \"5493aa92-3882-424c-b4dd-ab462198db29\") "
Oct 30 19:36:42.965817 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:42.965720    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:42.988693 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:42.930976    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5493aa92-3882-424c-b4dd-ab462198db29-maas-api-openshift-ca" pod "5493aa92-3882-424c-b4dd-ab462198db29" (UID: "5493aa92-3882-424c-b4dd-ab462198db29"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:36:43.129887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.124061    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5493aa92-3882-424c-b4dd-ab462198db29-kube-api-access-6mfwp" pod "5493aa92-3882-424c-b4dd-ab462198db29" (UID: "5493aa92-3882-424c-b4dd-ab462198db29"). InnerVolumeSpecName "kube-api-access-6mfwp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:36:43.130078 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.125839    5907 patch_prober.go:29] interesting pod/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 30 19:36:43.130231 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.130205    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2" podUID="5d44e37a-fe67-40e5-a3d4-c7665f508b76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 30 19:36:43.158988 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.134111    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5493aa92-3882-424c-b4dd-ab462198db29-maas-api-xks-ca" pod "5493aa92-3882-424c-b4dd-ab462198db29" (UID: "5493aa92-3882-424c-b4dd-ab462198db29"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:36:43.159176 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.125905    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Oct 30 19:36:43.159296 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.159271    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc" podUID="f67f5efb-73b5-49fa-b044-36471405ce25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Oct 30 19:36:43.159414 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.141844    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mfwp\" (UniqueName: \"kubernetes.io/projected/5493aa92-3882-424c-b4dd-ab462198db29-kube-api-access-6mfwp\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:43.159523 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.159511    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5493aa92-3882-424c-b4dd-ab462198db29-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:43.297892 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.297815    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5493aa92-3882-424c-b4dd-ab462198db29-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:43.313887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.307784    5907 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:43.457236 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.429516    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-38aef1fb-zvkv2"]
Oct 30 19:36:43.457236 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.430741    5907 cpu_manager.go:414] "containerMap: removing container" podUID="d99e5ef4-84b7-4dd7-9251-b3aac9842f97" containerName="revoke-keys"
Oct 30 19:36:43.457236 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.430764    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d99e5ef4-84b7-4dd7-9251-b3aac9842f97" containerName="revoke-keys"
Oct 30 19:36:43.457236 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.430821    5907 cpu_manager.go:414] "containerMap: removing container" podUID="5493aa92-3882-424c-b4dd-ab462198db29" containerName="revoke-keys"
Oct 30 19:36:43.457236 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.430830    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5493aa92-3882-424c-b4dd-ab462198db29" containerName="revoke-keys"
Oct 30 19:36:43.457236 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.430849    5907 cpu_manager.go:414] "containerMap: removing container" podUID="c0228ac8-d5d3-4f74-8505-71d58292814a" containerName="revoke-keys"
Oct 30 19:36:43.457236 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.430879    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c0228ac8-d5d3-4f74-8505-71d58292814a" containerName="revoke-keys"
Oct 30 19:36:43.457236 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.431038    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="d99e5ef4-84b7-4dd7-9251-b3aac9842f97" containerName="revoke-keys"
Oct 30 19:36:43.457236 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.431061    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="5493aa92-3882-424c-b4dd-ab462198db29" containerName="revoke-keys"
Oct 30 19:36:43.457236 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.431075    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="c0228ac8-d5d3-4f74-8505-71d58292814a" containerName="revoke-keys"
Oct 30 19:36:43.519908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.504263    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-38aef1fb-zvkv2"
Oct 30 19:36:43.520278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.520228    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:43.521790 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.521219    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": context deadline exceeded"
Oct 30 19:36:43.523021 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.522473    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 19:36:43.596885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.593021    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused" start-of-body=
Oct 30 19:36:43.596885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.593089    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8" podUID="3504b2cf-8f1c-453e-9372-3d425261b29c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused"
Oct 30 19:36:43.596885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.594953    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Oct 30 19:36:43.596885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.594997    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn" podUID="43db2eba-9244-4db6-ae67-8a20b01ea53e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Oct 30 19:36:43.955901 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.933457    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7fbf748e-xx9cn"
Oct 30 19:36:43.955901 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.934971    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7fbf748e-xx9cn" event={"ID":"5493aa92-3882-424c-b4dd-ab462198db29","Type":"ContainerDied","Data":"9a6e11c8fa2d3eefd24ba96e6125dbf9c0532bd2fa442a951c71e1022c6de03b"}
Oct 30 19:36:43.955901 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.935037    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9a6e11c8fa2d3eefd24ba96e6125dbf9c0532bd2fa442a951c71e1022c6de03b"
Oct 30 19:36:43.955901 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.938785    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-44b833ff-9zk88"
Oct 30 19:36:43.955901 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.955278    5907 patch_prober.go:29] interesting pod/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 30 19:36:43.955901 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.955335    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2" podUID="5d44e37a-fe67-40e5-a3d4-c7665f508b76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 30 19:36:43.955901 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.955891    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Oct 30 19:36:43.956687 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.955936    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc" podUID="f67f5efb-73b5-49fa-b044-36471405ce25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Oct 30 19:36:43.978920 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:43.965928    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-73ea5927-jmbxl"]
Oct 30 19:36:44.024521 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:44.024461    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:44.188040 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:44.126110    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-38aef1fb-zvkv2"]
Oct 30 19:36:44.188426 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:44.188400    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-73ea5927-jmbxl"]
Oct 30 19:36:44.188609 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:44.143789    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b347a63a-1983-4a46-aead-87af5078d844-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-38aef1fb-zvkv2\" (UID: \"b347a63a-1983-4a46-aead-87af5078d844\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-38aef1fb-zvkv2"
Oct 30 19:36:44.188774 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:44.188756    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phh6d\" (UniqueName: \"kubernetes.io/projected/b347a63a-1983-4a46-aead-87af5078d844-kube-api-access-phh6d\") pod \"maas-api-revoke-sub-38aef1fb-zvkv2\" (UID: \"b347a63a-1983-4a46-aead-87af5078d844\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-38aef1fb-zvkv2"
Oct 30 19:36:44.190745 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:44.126217    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-73ea5927-jmbxl"
Oct 30 19:36:44.190991 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:44.190966    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b347a63a-1983-4a46-aead-87af5078d844-maas-api-xks-ca\") pod \"maas-api-revoke-sub-38aef1fb-zvkv2\" (UID: \"b347a63a-1983-4a46-aead-87af5078d844\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-38aef1fb-zvkv2"
Oct 30 19:36:44.261080 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:44.261050    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:36:44.309322 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:44.309288    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/28e35538-2c5d-4ec4-8099-a7e7e6f8e270-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-73ea5927-jmbxl\" (UID: \"28e35538-2c5d-4ec4-8099-a7e7e6f8e270\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73ea5927-jmbxl"
Oct 30 19:36:44.309801 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:44.309776    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttksv\" (UniqueName: \"kubernetes.io/projected/28e35538-2c5d-4ec4-8099-a7e7e6f8e270-kube-api-access-ttksv\") pod \"maas-api-revoke-sub-73ea5927-jmbxl\" (UID: \"28e35538-2c5d-4ec4-8099-a7e7e6f8e270\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73ea5927-jmbxl"
Oct 30 19:36:44.311193 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:44.311166    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/28e35538-2c5d-4ec4-8099-a7e7e6f8e270-maas-api-xks-ca\") pod \"maas-api-revoke-sub-73ea5927-jmbxl\" (UID: \"28e35538-2c5d-4ec4-8099-a7e7e6f8e270\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73ea5927-jmbxl"
Oct 30 19:36:44.312153 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:44.312127    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b347a63a-1983-4a46-aead-87af5078d844-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-38aef1fb-zvkv2\" (UID: \"b347a63a-1983-4a46-aead-87af5078d844\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-38aef1fb-zvkv2"
Oct 30 19:36:44.314650 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:44.314628    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phh6d\" (UniqueName: \"kubernetes.io/projected/b347a63a-1983-4a46-aead-87af5078d844-kube-api-access-phh6d\") pod \"maas-api-revoke-sub-38aef1fb-zvkv2\" (UID: \"b347a63a-1983-4a46-aead-87af5078d844\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-38aef1fb-zvkv2"
Oct 30 19:36:44.315207 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:44.315179    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b347a63a-1983-4a46-aead-87af5078d844-maas-api-xks-ca\") pod \"maas-api-revoke-sub-38aef1fb-zvkv2\" (UID: \"b347a63a-1983-4a46-aead-87af5078d844\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-38aef1fb-zvkv2"
Oct 30 19:36:44.316147 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:44.314144    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b347a63a-1983-4a46-aead-87af5078d844-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-38aef1fb-zvkv2\" (UID: \"b347a63a-1983-4a46-aead-87af5078d844\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-38aef1fb-zvkv2"
Oct 30 19:36:44.375777 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:44.375741    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phh6d\" (UniqueName: \"kubernetes.io/projected/b347a63a-1983-4a46-aead-87af5078d844-kube-api-access-phh6d\") pod \"maas-api-revoke-sub-38aef1fb-zvkv2\" (UID: \"b347a63a-1983-4a46-aead-87af5078d844\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-38aef1fb-zvkv2"
Oct 30 19:36:44.399415 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:44.399345    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b347a63a-1983-4a46-aead-87af5078d844-maas-api-xks-ca\") pod \"maas-api-revoke-sub-38aef1fb-zvkv2\" (UID: \"b347a63a-1983-4a46-aead-87af5078d844\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-38aef1fb-zvkv2"
Oct 30 19:36:44.557000 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:44.513615    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Oct 30 19:36:44.557000 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:44.513680    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn" podUID="43db2eba-9244-4db6-ae67-8a20b01ea53e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Oct 30 19:36:44.557000 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:44.532525    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttksv\" (UniqueName: \"kubernetes.io/projected/28e35538-2c5d-4ec4-8099-a7e7e6f8e270-kube-api-access-ttksv\") pod \"maas-api-revoke-sub-73ea5927-jmbxl\" (UID: \"28e35538-2c5d-4ec4-8099-a7e7e6f8e270\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73ea5927-jmbxl"
Oct 30 19:36:44.557000 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:44.532606    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/28e35538-2c5d-4ec4-8099-a7e7e6f8e270-maas-api-xks-ca\") pod \"maas-api-revoke-sub-73ea5927-jmbxl\" (UID: \"28e35538-2c5d-4ec4-8099-a7e7e6f8e270\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73ea5927-jmbxl"
Oct 30 19:36:44.557000 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:44.547284    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:44.593380 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:44.592212    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttksv\" (UniqueName: \"kubernetes.io/projected/28e35538-2c5d-4ec4-8099-a7e7e6f8e270-kube-api-access-ttksv\") pod \"maas-api-revoke-sub-73ea5927-jmbxl\" (UID: \"28e35538-2c5d-4ec4-8099-a7e7e6f8e270\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73ea5927-jmbxl"
Oct 30 19:36:44.596184 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:44.594249    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/28e35538-2c5d-4ec4-8099-a7e7e6f8e270-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-73ea5927-jmbxl\" (UID: \"28e35538-2c5d-4ec4-8099-a7e7e6f8e270\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73ea5927-jmbxl"
Oct 30 19:36:44.647889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:44.599498    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:36:44.649411 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:44.622828    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused" start-of-body=
Oct 30 19:36:44.671877 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:44.636653    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-38aef1fb-zvkv2"
Oct 30 19:36:44.672198 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:44.671796    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8" podUID="3504b2cf-8f1c-453e-9372-3d425261b29c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused"
Oct 30 19:36:44.685473 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:44.532937    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/28e35538-2c5d-4ec4-8099-a7e7e6f8e270-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-73ea5927-jmbxl\" (UID: \"28e35538-2c5d-4ec4-8099-a7e7e6f8e270\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73ea5927-jmbxl"
Oct 30 19:36:44.920674 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:44.908547    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:36:44.994007 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:44.991067    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/28e35538-2c5d-4ec4-8099-a7e7e6f8e270-maas-api-xks-ca\") pod \"maas-api-revoke-sub-73ea5927-jmbxl\" (UID: \"28e35538-2c5d-4ec4-8099-a7e7e6f8e270\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73ea5927-jmbxl"
Oct 30 19:36:44.994007 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:44.991194    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Oct 30 19:36:44.994007 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:44.991236    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc" podUID="f67f5efb-73b5-49fa-b044-36471405ce25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Oct 30 19:36:44.994007 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:44.992761    5907 patch_prober.go:29] interesting pod/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 30 19:36:44.994007 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:44.992804    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2" podUID="5d44e37a-fe67-40e5-a3d4-c7665f508b76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 30 19:36:45.032094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.012780    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-eeeb8238-8kv2x"]
Oct 30 19:36:45.032094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.031280    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-73ea5927-jmbxl"
Oct 30 19:36:45.063490 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.063451    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:36:45.064170 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.063845    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-eeeb8238-8kv2x"
Oct 30 19:36:45.083574 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.067962    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-eeeb8238-8kv2x"]
Oct 30 19:36:45.102315 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.100772    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ce43f919-2435-4850-9436-3f7cc4863e4a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-eeeb8238-8kv2x\" (UID: \"ce43f919-2435-4850-9436-3f7cc4863e4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-eeeb8238-8kv2x"
Oct 30 19:36:45.102315 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.100898    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ce43f919-2435-4850-9436-3f7cc4863e4a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-eeeb8238-8kv2x\" (UID: \"ce43f919-2435-4850-9436-3f7cc4863e4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-eeeb8238-8kv2x"
Oct 30 19:36:45.102315 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.101052    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhdqj\" (UniqueName: \"kubernetes.io/projected/ce43f919-2435-4850-9436-3f7cc4863e4a-kube-api-access-hhdqj\") pod \"maas-api-revoke-sub-eeeb8238-8kv2x\" (UID: \"ce43f919-2435-4850-9436-3f7cc4863e4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-eeeb8238-8kv2x"
Oct 30 19:36:45.230729 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.213397    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhdqj\" (UniqueName: \"kubernetes.io/projected/ce43f919-2435-4850-9436-3f7cc4863e4a-kube-api-access-hhdqj\") pod \"maas-api-revoke-sub-eeeb8238-8kv2x\" (UID: \"ce43f919-2435-4850-9436-3f7cc4863e4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-eeeb8238-8kv2x"
Oct 30 19:36:45.230729 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.213632    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ce43f919-2435-4850-9436-3f7cc4863e4a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-eeeb8238-8kv2x\" (UID: \"ce43f919-2435-4850-9436-3f7cc4863e4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-eeeb8238-8kv2x"
Oct 30 19:36:45.230729 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.213717    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ce43f919-2435-4850-9436-3f7cc4863e4a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-eeeb8238-8kv2x\" (UID: \"ce43f919-2435-4850-9436-3f7cc4863e4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-eeeb8238-8kv2x"
Oct 30 19:36:45.230729 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.215249    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ce43f919-2435-4850-9436-3f7cc4863e4a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-eeeb8238-8kv2x\" (UID: \"ce43f919-2435-4850-9436-3f7cc4863e4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-eeeb8238-8kv2x"
Oct 30 19:36:45.237584 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.237514    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ce43f919-2435-4850-9436-3f7cc4863e4a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-eeeb8238-8kv2x\" (UID: \"ce43f919-2435-4850-9436-3f7cc4863e4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-eeeb8238-8kv2x"
Oct 30 19:36:45.244559 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.244494    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhdqj\" (UniqueName: \"kubernetes.io/projected/ce43f919-2435-4850-9436-3f7cc4863e4a-kube-api-access-hhdqj\") pod \"maas-api-revoke-sub-eeeb8238-8kv2x\" (UID: \"ce43f919-2435-4850-9436-3f7cc4863e4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-eeeb8238-8kv2x"
Oct 30 19:36:45.382942 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.378028    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e87b29b8-l6d7c"]
Oct 30 19:36:45.407823 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.407259    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e87b29b8-l6d7c"]
Oct 30 19:36:45.407823 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.407389    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e87b29b8-l6d7c"
Oct 30 19:36:45.444086 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.442783    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-eeeb8238-8kv2x"
Oct 30 19:36:45.500695 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.499187    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Oct 30 19:36:45.500695 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.499239    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn" podUID="43db2eba-9244-4db6-ae67-8a20b01ea53e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Oct 30 19:36:45.510910 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.508394    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-38aef1fb-zvkv2"]
Oct 30 19:36:45.547940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.547902    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwjrl\" (UniqueName: \"kubernetes.io/projected/27f2b187-607b-45f9-9d72-cbb2bfe1c81e-kube-api-access-gwjrl\") pod \"maas-api-revoke-sub-e87b29b8-l6d7c\" (UID: \"27f2b187-607b-45f9-9d72-cbb2bfe1c81e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e87b29b8-l6d7c"
Oct 30 19:36:45.548311 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.548264    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/27f2b187-607b-45f9-9d72-cbb2bfe1c81e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e87b29b8-l6d7c\" (UID: \"27f2b187-607b-45f9-9d72-cbb2bfe1c81e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e87b29b8-l6d7c"
Oct 30 19:36:45.548564 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.548492    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/27f2b187-607b-45f9-9d72-cbb2bfe1c81e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e87b29b8-l6d7c\" (UID: \"27f2b187-607b-45f9-9d72-cbb2bfe1c81e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e87b29b8-l6d7c"
Oct 30 19:36:45.553650 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.553558    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused" start-of-body=
Oct 30 19:36:45.553650 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.553603    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8" podUID="3504b2cf-8f1c-453e-9372-3d425261b29c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused"
Oct 30 19:36:45.557458 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.557046    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-73ea5927-jmbxl"]
Oct 30 19:36:45.651385 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.650712    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/27f2b187-607b-45f9-9d72-cbb2bfe1c81e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e87b29b8-l6d7c\" (UID: \"27f2b187-607b-45f9-9d72-cbb2bfe1c81e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e87b29b8-l6d7c"
Oct 30 19:36:45.651385 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.650818    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/27f2b187-607b-45f9-9d72-cbb2bfe1c81e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e87b29b8-l6d7c\" (UID: \"27f2b187-607b-45f9-9d72-cbb2bfe1c81e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e87b29b8-l6d7c"
Oct 30 19:36:45.654008 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.651876    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwjrl\" (UniqueName: \"kubernetes.io/projected/27f2b187-607b-45f9-9d72-cbb2bfe1c81e-kube-api-access-gwjrl\") pod \"maas-api-revoke-sub-e87b29b8-l6d7c\" (UID: \"27f2b187-607b-45f9-9d72-cbb2bfe1c81e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e87b29b8-l6d7c"
Oct 30 19:36:45.654567 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.654510    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/27f2b187-607b-45f9-9d72-cbb2bfe1c81e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e87b29b8-l6d7c\" (UID: \"27f2b187-607b-45f9-9d72-cbb2bfe1c81e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e87b29b8-l6d7c"
Oct 30 19:36:45.730310 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.719721    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/27f2b187-607b-45f9-9d72-cbb2bfe1c81e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e87b29b8-l6d7c\" (UID: \"27f2b187-607b-45f9-9d72-cbb2bfe1c81e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e87b29b8-l6d7c"
Oct 30 19:36:45.921979 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.896389    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwjrl\" (UniqueName: \"kubernetes.io/projected/27f2b187-607b-45f9-9d72-cbb2bfe1c81e-kube-api-access-gwjrl\") pod \"maas-api-revoke-sub-e87b29b8-l6d7c\" (UID: \"27f2b187-607b-45f9-9d72-cbb2bfe1c81e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e87b29b8-l6d7c"
Oct 30 19:36:45.965886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.961849    5907 patch_prober.go:29] interesting pod/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 30 19:36:45.965886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.962055    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2" podUID="5d44e37a-fe67-40e5-a3d4-c7665f508b76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 30 19:36:45.965886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.962510    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Oct 30 19:36:45.965886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:45.962547    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc" podUID="f67f5efb-73b5-49fa-b044-36471405ce25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Oct 30 19:36:46.037305 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:46.037219    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-38aef1fb-zvkv2" event={"ID":"b347a63a-1983-4a46-aead-87af5078d844","Type":"ContainerStarted","Data":"e8b116e617e56271f58c296f85692b7f4419fa5dc5a76e6ff411e297ddcffacb"}
Oct 30 19:36:46.040556 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:46.040529    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-73ea5927-jmbxl" event={"ID":"28e35538-2c5d-4ec4-8099-a7e7e6f8e270","Type":"ContainerStarted","Data":"101ead1759f20d38d2b5f0cbe2589d35d4dcc49684504232ef20ad6cbcb7798c"}
Oct 30 19:36:46.081267 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:46.081119    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e87b29b8-l6d7c"
Oct 30 19:36:46.447901 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:46.427619    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-eeeb8238-8kv2x"]
Oct 30 19:36:46.652580 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:46.651300    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"
Oct 30 19:36:46.652755 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:46.652717    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"
Oct 30 19:36:46.655406 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:46.654968    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"
Oct 30 19:36:46.810453 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:46.803820    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"
Oct 30 19:36:46.810453 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:46.805714    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"
Oct 30 19:36:46.825306 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:46.823346    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"
Oct 30 19:36:46.986655 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:46.984836    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e87b29b8-l6d7c"]
Oct 30 19:36:47.000328 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:36:47.000292    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27f2b187_607b_45f9_9d72_cbb2bfe1c81e.slice/crio-a9b8d7a642a3d59b9f82f4d611a46eedb097d77fb675dca3fbb1a938765250b6 WatchSource:0}: Error finding container a9b8d7a642a3d59b9f82f4d611a46eedb097d77fb675dca3fbb1a938765250b6: Status 404 returned error can't find the container with id a9b8d7a642a3d59b9f82f4d611a46eedb097d77fb675dca3fbb1a938765250b6
Oct 30 19:36:47.068944 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:47.065784    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-38aef1fb-zvkv2" event={"ID":"b347a63a-1983-4a46-aead-87af5078d844","Type":"ContainerStarted","Data":"123f3de9c7f069ff2b3f320ea8e6499f343b0d355ae8df6f5a86d880325cab24"}
Oct 30 19:36:47.070045 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:47.070016    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-eeeb8238-8kv2x" event={"ID":"ce43f919-2435-4850-9436-3f7cc4863e4a","Type":"ContainerStarted","Data":"605e663001b21092383ab253c830b8c26839fb9e1edfdd7a651117546d9c02e1"}
Oct 30 19:36:47.071855 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:47.071833    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-eeeb8238-8kv2x" event={"ID":"ce43f919-2435-4850-9436-3f7cc4863e4a","Type":"ContainerStarted","Data":"07926ccd87778b92707167dbf9bf995526e4fb2a9fc24e6be61859763bc32bef"}
Oct 30 19:36:47.073749 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:47.073049    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-73ea5927-jmbxl" event={"ID":"28e35538-2c5d-4ec4-8099-a7e7e6f8e270","Type":"ContainerStarted","Data":"66ed25c2651916d246bac929ff924f7fa288a9b88a47da69c885cc6e058717d1"}
Oct 30 19:36:47.076583 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:47.076000    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e87b29b8-l6d7c" event={"ID":"27f2b187-607b-45f9-9d72-cbb2bfe1c81e","Type":"ContainerStarted","Data":"a9b8d7a642a3d59b9f82f4d611a46eedb097d77fb675dca3fbb1a938765250b6"}
Oct 30 19:36:47.111512 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:47.111452    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-38aef1fb-zvkv2" podStartSLOduration=5.11143408 podStartE2EDuration="5.11143408s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:36:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:36:47.107101281 +0000 UTC m=+918.950033233" watchObservedRunningTime="2026-10-30 19:36:47.11143408 +0000 UTC m=+918.954366021"
Oct 30 19:36:47.149177 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:47.149128    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-eeeb8238-8kv2x" podStartSLOduration=3.149114403 podStartE2EDuration="3.149114403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:36:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:36:47.148338408 +0000 UTC m=+918.991270371" watchObservedRunningTime="2026-10-30 19:36:47.149114403 +0000 UTC m=+918.992046342"
Oct 30 19:36:47.193317 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:47.193253    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-73ea5927-jmbxl" podStartSLOduration=5.193232447 podStartE2EDuration="5.193232447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:36:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:36:47.190058682 +0000 UTC m=+919.032990621" watchObservedRunningTime="2026-10-30 19:36:47.193232447 +0000 UTC m=+919.036164387"
Oct 30 19:36:47.215888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:47.215720    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"
Oct 30 19:36:47.216780 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:47.216754    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"
Oct 30 19:36:47.217811 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:47.217787    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"
Oct 30 19:36:47.314277 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:47.314231    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"
Oct 30 19:36:47.315322 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:47.315163    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"
Oct 30 19:36:47.316700 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:47.316680    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"
Oct 30 19:36:47.504897 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:36:47.504838    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:36:48.086805 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:48.086722    5907 generic.go:356] "Generic (PLEG): container finished" podID="ce43f919-2435-4850-9436-3f7cc4863e4a" containerID="605e663001b21092383ab253c830b8c26839fb9e1edfdd7a651117546d9c02e1" exitCode=0
Oct 30 19:36:48.087319 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:48.086808    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-eeeb8238-8kv2x" event={"ID":"ce43f919-2435-4850-9436-3f7cc4863e4a","Type":"ContainerDied","Data":"605e663001b21092383ab253c830b8c26839fb9e1edfdd7a651117546d9c02e1"}
Oct 30 19:36:48.088528 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:48.088503    5907 generic.go:356] "Generic (PLEG): container finished" podID="28e35538-2c5d-4ec4-8099-a7e7e6f8e270" containerID="66ed25c2651916d246bac929ff924f7fa288a9b88a47da69c885cc6e058717d1" exitCode=0
Oct 30 19:36:48.088669 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:48.088584    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-73ea5927-jmbxl" event={"ID":"28e35538-2c5d-4ec4-8099-a7e7e6f8e270","Type":"ContainerDied","Data":"66ed25c2651916d246bac929ff924f7fa288a9b88a47da69c885cc6e058717d1"}
Oct 30 19:36:48.090314 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:48.090294    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e87b29b8-l6d7c" event={"ID":"27f2b187-607b-45f9-9d72-cbb2bfe1c81e","Type":"ContainerStarted","Data":"454adf472db5f76eb43e65fa2f344d81bd761a4fdf2696fe788be676351256c0"}
Oct 30 19:36:48.092146 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:48.092117    5907 generic.go:356] "Generic (PLEG): container finished" podID="b347a63a-1983-4a46-aead-87af5078d844" containerID="123f3de9c7f069ff2b3f320ea8e6499f343b0d355ae8df6f5a86d880325cab24" exitCode=0
Oct 30 19:36:48.092233 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:48.092189    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-38aef1fb-zvkv2" event={"ID":"b347a63a-1983-4a46-aead-87af5078d844","Type":"ContainerDied","Data":"123f3de9c7f069ff2b3f320ea8e6499f343b0d355ae8df6f5a86d880325cab24"}
Oct 30 19:36:48.211442 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:48.211395    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e87b29b8-l6d7c" podStartSLOduration=3.211381209 podStartE2EDuration="3.211381209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:36:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:36:48.205656786 +0000 UTC m=+920.048588735" watchObservedRunningTime="2026-10-30 19:36:48.211381209 +0000 UTC m=+920.054313148"
Oct 30 19:36:49.103307 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:49.103263    5907 generic.go:356] "Generic (PLEG): container finished" podID="27f2b187-607b-45f9-9d72-cbb2bfe1c81e" containerID="454adf472db5f76eb43e65fa2f344d81bd761a4fdf2696fe788be676351256c0" exitCode=0
Oct 30 19:36:49.103853 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:49.103504    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e87b29b8-l6d7c" event={"ID":"27f2b187-607b-45f9-9d72-cbb2bfe1c81e","Type":"ContainerDied","Data":"454adf472db5f76eb43e65fa2f344d81bd761a4fdf2696fe788be676351256c0"}
Oct 30 19:36:49.341616 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:49.341584    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-38aef1fb-zvkv2"
Oct 30 19:36:49.382046 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:49.382018    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-73ea5927-jmbxl"
Oct 30 19:36:49.387215 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:49.387191    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-eeeb8238-8kv2x"
Oct 30 19:36:49.430127 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:49.430091    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/28e35538-2c5d-4ec4-8099-a7e7e6f8e270-kube-api-access-ttksv\" (UniqueName: \"kubernetes.io/projected/28e35538-2c5d-4ec4-8099-a7e7e6f8e270-kube-api-access-ttksv\") pod \"28e35538-2c5d-4ec4-8099-a7e7e6f8e270\" (UID: \"28e35538-2c5d-4ec4-8099-a7e7e6f8e270\") "
Oct 30 19:36:49.430313 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:49.430142    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/28e35538-2c5d-4ec4-8099-a7e7e6f8e270-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/28e35538-2c5d-4ec4-8099-a7e7e6f8e270-maas-api-xks-ca\") pod \"28e35538-2c5d-4ec4-8099-a7e7e6f8e270\" (UID: \"28e35538-2c5d-4ec4-8099-a7e7e6f8e270\") "
Oct 30 19:36:49.430313 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:49.430212    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b347a63a-1983-4a46-aead-87af5078d844-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b347a63a-1983-4a46-aead-87af5078d844-maas-api-openshift-ca\") pod \"b347a63a-1983-4a46-aead-87af5078d844\" (UID: \"b347a63a-1983-4a46-aead-87af5078d844\") "
Oct 30 19:36:49.431048 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:49.430685    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ce43f919-2435-4850-9436-3f7cc4863e4a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ce43f919-2435-4850-9436-3f7cc4863e4a-maas-api-xks-ca\") pod \"ce43f919-2435-4850-9436-3f7cc4863e4a\" (UID: \"ce43f919-2435-4850-9436-3f7cc4863e4a\") "
Oct 30 19:36:49.431048 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:49.430964    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ce43f919-2435-4850-9436-3f7cc4863e4a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ce43f919-2435-4850-9436-3f7cc4863e4a-maas-api-openshift-ca\") pod \"ce43f919-2435-4850-9436-3f7cc4863e4a\" (UID: \"ce43f919-2435-4850-9436-3f7cc4863e4a\") "
Oct 30 19:36:49.431048 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:49.431050    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ce43f919-2435-4850-9436-3f7cc4863e4a-kube-api-access-hhdqj\" (UniqueName: \"kubernetes.io/projected/ce43f919-2435-4850-9436-3f7cc4863e4a-kube-api-access-hhdqj\") pod \"ce43f919-2435-4850-9436-3f7cc4863e4a\" (UID: \"ce43f919-2435-4850-9436-3f7cc4863e4a\") "
Oct 30 19:36:49.431207 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:49.431152    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b347a63a-1983-4a46-aead-87af5078d844-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b347a63a-1983-4a46-aead-87af5078d844-maas-api-xks-ca\") pod \"b347a63a-1983-4a46-aead-87af5078d844\" (UID: \"b347a63a-1983-4a46-aead-87af5078d844\") "
Oct 30 19:36:49.431287 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:49.431271    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/28e35538-2c5d-4ec4-8099-a7e7e6f8e270-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/28e35538-2c5d-4ec4-8099-a7e7e6f8e270-maas-api-openshift-ca\") pod \"28e35538-2c5d-4ec4-8099-a7e7e6f8e270\" (UID: \"28e35538-2c5d-4ec4-8099-a7e7e6f8e270\") "
Oct 30 19:36:49.431366 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:49.431337    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b347a63a-1983-4a46-aead-87af5078d844-kube-api-access-phh6d\" (UniqueName: \"kubernetes.io/projected/b347a63a-1983-4a46-aead-87af5078d844-kube-api-access-phh6d\") pod \"b347a63a-1983-4a46-aead-87af5078d844\" (UID: \"b347a63a-1983-4a46-aead-87af5078d844\") "
Oct 30 19:36:49.432577 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:49.432311    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce43f919-2435-4850-9436-3f7cc4863e4a-maas-api-openshift-ca" pod "ce43f919-2435-4850-9436-3f7cc4863e4a" (UID: "ce43f919-2435-4850-9436-3f7cc4863e4a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:36:49.433648 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:49.433619    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b347a63a-1983-4a46-aead-87af5078d844-maas-api-openshift-ca" pod "b347a63a-1983-4a46-aead-87af5078d844" (UID: "b347a63a-1983-4a46-aead-87af5078d844"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:36:49.433836 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:49.433811    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28e35538-2c5d-4ec4-8099-a7e7e6f8e270-maas-api-openshift-ca" pod "28e35538-2c5d-4ec4-8099-a7e7e6f8e270" (UID: "28e35538-2c5d-4ec4-8099-a7e7e6f8e270"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:36:49.434812 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:49.434786    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce43f919-2435-4850-9436-3f7cc4863e4a-kube-api-access-hhdqj" pod "ce43f919-2435-4850-9436-3f7cc4863e4a" (UID: "ce43f919-2435-4850-9436-3f7cc4863e4a"). InnerVolumeSpecName "kube-api-access-hhdqj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:36:49.435037 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:49.435003    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b347a63a-1983-4a46-aead-87af5078d844-kube-api-access-phh6d" pod "b347a63a-1983-4a46-aead-87af5078d844" (UID: "b347a63a-1983-4a46-aead-87af5078d844"). InnerVolumeSpecName "kube-api-access-phh6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:36:49.435604 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:49.435580    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28e35538-2c5d-4ec4-8099-a7e7e6f8e270-kube-api-access-ttksv" pod "28e35538-2c5d-4ec4-8099-a7e7e6f8e270" (UID: "28e35538-2c5d-4ec4-8099-a7e7e6f8e270"). InnerVolumeSpecName "kube-api-access-ttksv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:36:49.436046 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:49.436016    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce43f919-2435-4850-9436-3f7cc4863e4a-maas-api-xks-ca" pod "ce43f919-2435-4850-9436-3f7cc4863e4a" (UID: "ce43f919-2435-4850-9436-3f7cc4863e4a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:36:49.436148 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:49.436075    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28e35538-2c5d-4ec4-8099-a7e7e6f8e270-maas-api-xks-ca" pod "28e35538-2c5d-4ec4-8099-a7e7e6f8e270" (UID: "28e35538-2c5d-4ec4-8099-a7e7e6f8e270"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:36:49.436210 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:49.436186    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b347a63a-1983-4a46-aead-87af5078d844-maas-api-xks-ca" pod "b347a63a-1983-4a46-aead-87af5078d844" (UID: "b347a63a-1983-4a46-aead-87af5078d844"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:36:49.533819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:49.533786    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phh6d\" (UniqueName: \"kubernetes.io/projected/b347a63a-1983-4a46-aead-87af5078d844-kube-api-access-phh6d\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:49.533819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:49.533816    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ttksv\" (UniqueName: \"kubernetes.io/projected/28e35538-2c5d-4ec4-8099-a7e7e6f8e270-kube-api-access-ttksv\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:49.534000 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:49.533827    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/28e35538-2c5d-4ec4-8099-a7e7e6f8e270-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:49.534000 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:49.533836    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b347a63a-1983-4a46-aead-87af5078d844-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:49.534000 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:49.533845    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ce43f919-2435-4850-9436-3f7cc4863e4a-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:49.534000 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:49.533853    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ce43f919-2435-4850-9436-3f7cc4863e4a-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:49.534000 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:49.533875    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhdqj\" (UniqueName: \"kubernetes.io/projected/ce43f919-2435-4850-9436-3f7cc4863e4a-kube-api-access-hhdqj\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:49.534000 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:49.533883    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b347a63a-1983-4a46-aead-87af5078d844-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:49.534000 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:49.533893    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/28e35538-2c5d-4ec4-8099-a7e7e6f8e270-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:50.106134 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.106102    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-95a4820f-ng97l"]
Oct 30 19:36:50.106889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.106871    5907 cpu_manager.go:414] "containerMap: removing container" podUID="ce43f919-2435-4850-9436-3f7cc4863e4a" containerName="revoke-keys"
Oct 30 19:36:50.106889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.106889    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ce43f919-2435-4850-9436-3f7cc4863e4a" containerName="revoke-keys"
Oct 30 19:36:50.107020 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.106908    5907 cpu_manager.go:414] "containerMap: removing container" podUID="28e35538-2c5d-4ec4-8099-a7e7e6f8e270" containerName="revoke-keys"
Oct 30 19:36:50.107020 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.106914    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="28e35538-2c5d-4ec4-8099-a7e7e6f8e270" containerName="revoke-keys"
Oct 30 19:36:50.107020 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.106972    5907 cpu_manager.go:414] "containerMap: removing container" podUID="b347a63a-1983-4a46-aead-87af5078d844" containerName="revoke-keys"
Oct 30 19:36:50.107020 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.106978    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b347a63a-1983-4a46-aead-87af5078d844" containerName="revoke-keys"
Oct 30 19:36:50.107224 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.107092    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="28e35538-2c5d-4ec4-8099-a7e7e6f8e270" containerName="revoke-keys"
Oct 30 19:36:50.107224 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.107108    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="b347a63a-1983-4a46-aead-87af5078d844" containerName="revoke-keys"
Oct 30 19:36:50.107224 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.107116    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="ce43f919-2435-4850-9436-3f7cc4863e4a" containerName="revoke-keys"
Oct 30 19:36:50.112216 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.112193    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-95a4820f-ng97l"
Oct 30 19:36:50.114814 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.114797    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-38aef1fb-zvkv2"
Oct 30 19:36:50.114947 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.114832    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-38aef1fb-zvkv2" event={"ID":"b347a63a-1983-4a46-aead-87af5078d844","Type":"ContainerDied","Data":"e8b116e617e56271f58c296f85692b7f4419fa5dc5a76e6ff411e297ddcffacb"}
Oct 30 19:36:50.114947 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.114894    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e8b116e617e56271f58c296f85692b7f4419fa5dc5a76e6ff411e297ddcffacb"
Oct 30 19:36:50.116967 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.116948    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-eeeb8238-8kv2x"
Oct 30 19:36:50.117074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.116997    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-eeeb8238-8kv2x" event={"ID":"ce43f919-2435-4850-9436-3f7cc4863e4a","Type":"ContainerDied","Data":"07926ccd87778b92707167dbf9bf995526e4fb2a9fc24e6be61859763bc32bef"}
Oct 30 19:36:50.117074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.117042    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="07926ccd87778b92707167dbf9bf995526e4fb2a9fc24e6be61859763bc32bef"
Oct 30 19:36:50.118721 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.118704    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-73ea5927-jmbxl"
Oct 30 19:36:50.118814 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.118747    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-73ea5927-jmbxl" event={"ID":"28e35538-2c5d-4ec4-8099-a7e7e6f8e270","Type":"ContainerDied","Data":"101ead1759f20d38d2b5f0cbe2589d35d4dcc49684504232ef20ad6cbcb7798c"}
Oct 30 19:36:50.118814 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.118784    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="101ead1759f20d38d2b5f0cbe2589d35d4dcc49684504232ef20ad6cbcb7798c"
Oct 30 19:36:50.133947 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.133924    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-95a4820f-ng97l"]
Oct 30 19:36:50.143935 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.143899    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj5pq\" (UniqueName: \"kubernetes.io/projected/dc5fa222-f6d2-4a77-9cc9-8a70bd322c95-kube-api-access-hj5pq\") pod \"maas-api-revoke-sub-95a4820f-ng97l\" (UID: \"dc5fa222-f6d2-4a77-9cc9-8a70bd322c95\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-95a4820f-ng97l"
Oct 30 19:36:50.145114 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.145081    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dc5fa222-f6d2-4a77-9cc9-8a70bd322c95-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-95a4820f-ng97l\" (UID: \"dc5fa222-f6d2-4a77-9cc9-8a70bd322c95\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-95a4820f-ng97l"
Oct 30 19:36:50.145311 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.145286    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dc5fa222-f6d2-4a77-9cc9-8a70bd322c95-maas-api-xks-ca\") pod \"maas-api-revoke-sub-95a4820f-ng97l\" (UID: \"dc5fa222-f6d2-4a77-9cc9-8a70bd322c95\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-95a4820f-ng97l"
Oct 30 19:36:50.250802 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.250761    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dc5fa222-f6d2-4a77-9cc9-8a70bd322c95-maas-api-xks-ca\") pod \"maas-api-revoke-sub-95a4820f-ng97l\" (UID: \"dc5fa222-f6d2-4a77-9cc9-8a70bd322c95\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-95a4820f-ng97l"
Oct 30 19:36:50.251663 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.251632    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hj5pq\" (UniqueName: \"kubernetes.io/projected/dc5fa222-f6d2-4a77-9cc9-8a70bd322c95-kube-api-access-hj5pq\") pod \"maas-api-revoke-sub-95a4820f-ng97l\" (UID: \"dc5fa222-f6d2-4a77-9cc9-8a70bd322c95\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-95a4820f-ng97l"
Oct 30 19:36:50.252125 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.252102    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dc5fa222-f6d2-4a77-9cc9-8a70bd322c95-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-95a4820f-ng97l\" (UID: \"dc5fa222-f6d2-4a77-9cc9-8a70bd322c95\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-95a4820f-ng97l"
Oct 30 19:36:50.252733 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.252709    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dc5fa222-f6d2-4a77-9cc9-8a70bd322c95-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-95a4820f-ng97l\" (UID: \"dc5fa222-f6d2-4a77-9cc9-8a70bd322c95\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-95a4820f-ng97l"
Oct 30 19:36:50.253381 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.253359    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dc5fa222-f6d2-4a77-9cc9-8a70bd322c95-maas-api-xks-ca\") pod \"maas-api-revoke-sub-95a4820f-ng97l\" (UID: \"dc5fa222-f6d2-4a77-9cc9-8a70bd322c95\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-95a4820f-ng97l"
Oct 30 19:36:50.272343 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.272320    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj5pq\" (UniqueName: \"kubernetes.io/projected/dc5fa222-f6d2-4a77-9cc9-8a70bd322c95-kube-api-access-hj5pq\") pod \"maas-api-revoke-sub-95a4820f-ng97l\" (UID: \"dc5fa222-f6d2-4a77-9cc9-8a70bd322c95\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-95a4820f-ng97l"
Oct 30 19:36:50.311907 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.311887    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e87b29b8-l6d7c"
Oct 30 19:36:50.353849 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.353816    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/27f2b187-607b-45f9-9d72-cbb2bfe1c81e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/27f2b187-607b-45f9-9d72-cbb2bfe1c81e-maas-api-openshift-ca\") pod \"27f2b187-607b-45f9-9d72-cbb2bfe1c81e\" (UID: \"27f2b187-607b-45f9-9d72-cbb2bfe1c81e\") "
Oct 30 19:36:50.354091 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.353895    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/27f2b187-607b-45f9-9d72-cbb2bfe1c81e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/27f2b187-607b-45f9-9d72-cbb2bfe1c81e-maas-api-xks-ca\") pod \"27f2b187-607b-45f9-9d72-cbb2bfe1c81e\" (UID: \"27f2b187-607b-45f9-9d72-cbb2bfe1c81e\") "
Oct 30 19:36:50.354091 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.354033    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/27f2b187-607b-45f9-9d72-cbb2bfe1c81e-kube-api-access-gwjrl\" (UniqueName: \"kubernetes.io/projected/27f2b187-607b-45f9-9d72-cbb2bfe1c81e-kube-api-access-gwjrl\") pod \"27f2b187-607b-45f9-9d72-cbb2bfe1c81e\" (UID: \"27f2b187-607b-45f9-9d72-cbb2bfe1c81e\") "
Oct 30 19:36:50.358032 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.354249    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27f2b187-607b-45f9-9d72-cbb2bfe1c81e-maas-api-openshift-ca" pod "27f2b187-607b-45f9-9d72-cbb2bfe1c81e" (UID: "27f2b187-607b-45f9-9d72-cbb2bfe1c81e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:36:50.358032 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.355013    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/27f2b187-607b-45f9-9d72-cbb2bfe1c81e-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:50.358305 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.358283    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27f2b187-607b-45f9-9d72-cbb2bfe1c81e-maas-api-xks-ca" pod "27f2b187-607b-45f9-9d72-cbb2bfe1c81e" (UID: "27f2b187-607b-45f9-9d72-cbb2bfe1c81e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:36:50.359257 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.359230    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27f2b187-607b-45f9-9d72-cbb2bfe1c81e-kube-api-access-gwjrl" pod "27f2b187-607b-45f9-9d72-cbb2bfe1c81e" (UID: "27f2b187-607b-45f9-9d72-cbb2bfe1c81e"). InnerVolumeSpecName "kube-api-access-gwjrl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:36:50.442647 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.442608    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-95a4820f-ng97l"
Oct 30 19:36:50.457066 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.457042    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/27f2b187-607b-45f9-9d72-cbb2bfe1c81e-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:50.457066 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.457066    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwjrl\" (UniqueName: \"kubernetes.io/projected/27f2b187-607b-45f9-9d72-cbb2bfe1c81e-kube-api-access-gwjrl\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:50.762312 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:36:50.762277    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc5fa222_f6d2_4a77_9cc9_8a70bd322c95.slice/crio-a5d1618aa39d256124bb306e08caf542bfbbb52279d4d00a4dec0ae4183bf630 WatchSource:0}: Error finding container a5d1618aa39d256124bb306e08caf542bfbbb52279d4d00a4dec0ae4183bf630: Status 404 returned error can't find the container with id a5d1618aa39d256124bb306e08caf542bfbbb52279d4d00a4dec0ae4183bf630
Oct 30 19:36:50.763522 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:50.763494    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-95a4820f-ng97l"]
Oct 30 19:36:51.129205 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:51.129168    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e87b29b8-l6d7c" event={"ID":"27f2b187-607b-45f9-9d72-cbb2bfe1c81e","Type":"ContainerDied","Data":"a9b8d7a642a3d59b9f82f4d611a46eedb097d77fb675dca3fbb1a938765250b6"}
Oct 30 19:36:51.129673 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:51.129230    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a9b8d7a642a3d59b9f82f4d611a46eedb097d77fb675dca3fbb1a938765250b6"
Oct 30 19:36:51.129673 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:51.129180    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e87b29b8-l6d7c"
Oct 30 19:36:51.131119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:51.131081    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-95a4820f-ng97l" event={"ID":"dc5fa222-f6d2-4a77-9cc9-8a70bd322c95","Type":"ContainerStarted","Data":"198c1d15feb8796cedd11964f77482afd070de6edd30b95afe6c66e6e5ab3996"}
Oct 30 19:36:51.131245 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:51.131138    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-95a4820f-ng97l" event={"ID":"dc5fa222-f6d2-4a77-9cc9-8a70bd322c95","Type":"ContainerStarted","Data":"a5d1618aa39d256124bb306e08caf542bfbbb52279d4d00a4dec0ae4183bf630"}
Oct 30 19:36:51.204886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:51.202848    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-95a4820f-ng97l" podStartSLOduration=1.202827916 podStartE2EDuration="1.202827916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:36:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:36:51.202432076 +0000 UTC m=+923.045364016" watchObservedRunningTime="2026-10-30 19:36:51.202827916 +0000 UTC m=+923.045759856"
Oct 30 19:36:52.142089 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:52.142056    5907 generic.go:356] "Generic (PLEG): container finished" podID="dc5fa222-f6d2-4a77-9cc9-8a70bd322c95" containerID="198c1d15feb8796cedd11964f77482afd070de6edd30b95afe6c66e6e5ab3996" exitCode=0
Oct 30 19:36:52.142535 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:52.142146    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-95a4820f-ng97l" event={"ID":"dc5fa222-f6d2-4a77-9cc9-8a70bd322c95","Type":"ContainerDied","Data":"198c1d15feb8796cedd11964f77482afd070de6edd30b95afe6c66e6e5ab3996"}
Oct 30 19:36:53.206607 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.206571    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c8b8d0ba-n2hgt"]
Oct 30 19:36:53.207381 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.207363    5907 cpu_manager.go:414] "containerMap: removing container" podUID="27f2b187-607b-45f9-9d72-cbb2bfe1c81e" containerName="revoke-keys"
Oct 30 19:36:53.207425 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.207383    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="27f2b187-607b-45f9-9d72-cbb2bfe1c81e" containerName="revoke-keys"
Oct 30 19:36:53.207550 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.207541    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="27f2b187-607b-45f9-9d72-cbb2bfe1c81e" containerName="revoke-keys"
Oct 30 19:36:53.211896 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.211837    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c8b8d0ba-n2hgt"
Oct 30 19:36:53.245738 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.242915    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8e42d71a-q6p8w"]
Oct 30 19:36:53.255896 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.252040    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8e42d71a-q6p8w"
Oct 30 19:36:53.284211 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.284177    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c8b8d0ba-n2hgt"]
Oct 30 19:36:53.294598 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.294572    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8e42d71a-q6p8w"]
Oct 30 19:36:53.342475 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.342445    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t68bp\" (UniqueName: \"kubernetes.io/projected/97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5-kube-api-access-t68bp\") pod \"maas-api-revoke-sub-c8b8d0ba-n2hgt\" (UID: \"97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c8b8d0ba-n2hgt"
Oct 30 19:36:53.342630 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.342605    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c8b8d0ba-n2hgt\" (UID: \"97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c8b8d0ba-n2hgt"
Oct 30 19:36:53.342698 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.342668    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/04d55fb9-1145-4041-bb56-e7b398104a74-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8e42d71a-q6p8w\" (UID: \"04d55fb9-1145-4041-bb56-e7b398104a74\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8e42d71a-q6p8w"
Oct 30 19:36:53.342810 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.342791    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/04d55fb9-1145-4041-bb56-e7b398104a74-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8e42d71a-q6p8w\" (UID: \"04d55fb9-1145-4041-bb56-e7b398104a74\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8e42d71a-q6p8w"
Oct 30 19:36:53.342883 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.342825    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92k7m\" (UniqueName: \"kubernetes.io/projected/04d55fb9-1145-4041-bb56-e7b398104a74-kube-api-access-92k7m\") pod \"maas-api-revoke-sub-8e42d71a-q6p8w\" (UID: \"04d55fb9-1145-4041-bb56-e7b398104a74\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8e42d71a-q6p8w"
Oct 30 19:36:53.342945 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.342921    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c8b8d0ba-n2hgt\" (UID: \"97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c8b8d0ba-n2hgt"
Oct 30 19:36:53.345890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.345841    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-95a4820f-ng97l"
Oct 30 19:36:53.444491 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.444412    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc5fa222-f6d2-4a77-9cc9-8a70bd322c95-kube-api-access-hj5pq\" (UniqueName: \"kubernetes.io/projected/dc5fa222-f6d2-4a77-9cc9-8a70bd322c95-kube-api-access-hj5pq\") pod \"dc5fa222-f6d2-4a77-9cc9-8a70bd322c95\" (UID: \"dc5fa222-f6d2-4a77-9cc9-8a70bd322c95\") "
Oct 30 19:36:53.444651 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.444588    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dc5fa222-f6d2-4a77-9cc9-8a70bd322c95-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dc5fa222-f6d2-4a77-9cc9-8a70bd322c95-maas-api-xks-ca\") pod \"dc5fa222-f6d2-4a77-9cc9-8a70bd322c95\" (UID: \"dc5fa222-f6d2-4a77-9cc9-8a70bd322c95\") "
Oct 30 19:36:53.444719 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.444679    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dc5fa222-f6d2-4a77-9cc9-8a70bd322c95-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dc5fa222-f6d2-4a77-9cc9-8a70bd322c95-maas-api-openshift-ca\") pod \"dc5fa222-f6d2-4a77-9cc9-8a70bd322c95\" (UID: \"dc5fa222-f6d2-4a77-9cc9-8a70bd322c95\") "
Oct 30 19:36:53.445118 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.445090    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t68bp\" (UniqueName: \"kubernetes.io/projected/97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5-kube-api-access-t68bp\") pod \"maas-api-revoke-sub-c8b8d0ba-n2hgt\" (UID: \"97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c8b8d0ba-n2hgt"
Oct 30 19:36:53.445435 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.445132    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc5fa222-f6d2-4a77-9cc9-8a70bd322c95-maas-api-openshift-ca" pod "dc5fa222-f6d2-4a77-9cc9-8a70bd322c95" (UID: "dc5fa222-f6d2-4a77-9cc9-8a70bd322c95"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:36:53.445435 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.445389    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c8b8d0ba-n2hgt\" (UID: \"97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c8b8d0ba-n2hgt"
Oct 30 19:36:53.445581 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.445492    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/04d55fb9-1145-4041-bb56-e7b398104a74-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8e42d71a-q6p8w\" (UID: \"04d55fb9-1145-4041-bb56-e7b398104a74\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8e42d71a-q6p8w"
Oct 30 19:36:53.445646 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.445615    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/04d55fb9-1145-4041-bb56-e7b398104a74-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8e42d71a-q6p8w\" (UID: \"04d55fb9-1145-4041-bb56-e7b398104a74\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8e42d71a-q6p8w"
Oct 30 19:36:53.445712 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.445668    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92k7m\" (UniqueName: \"kubernetes.io/projected/04d55fb9-1145-4041-bb56-e7b398104a74-kube-api-access-92k7m\") pod \"maas-api-revoke-sub-8e42d71a-q6p8w\" (UID: \"04d55fb9-1145-4041-bb56-e7b398104a74\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8e42d71a-q6p8w"
Oct 30 19:36:53.445772 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.445723    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c8b8d0ba-n2hgt\" (UID: \"97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c8b8d0ba-n2hgt"
Oct 30 19:36:53.446001 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.445972    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dc5fa222-f6d2-4a77-9cc9-8a70bd322c95-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:53.446980 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.446956    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c8b8d0ba-n2hgt\" (UID: \"97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c8b8d0ba-n2hgt"
Oct 30 19:36:53.447278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.447253    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc5fa222-f6d2-4a77-9cc9-8a70bd322c95-kube-api-access-hj5pq" pod "dc5fa222-f6d2-4a77-9cc9-8a70bd322c95" (UID: "dc5fa222-f6d2-4a77-9cc9-8a70bd322c95"). InnerVolumeSpecName "kube-api-access-hj5pq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:36:53.447400 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.447336    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/04d55fb9-1145-4041-bb56-e7b398104a74-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8e42d71a-q6p8w\" (UID: \"04d55fb9-1145-4041-bb56-e7b398104a74\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8e42d71a-q6p8w"
Oct 30 19:36:53.447675 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.447651    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc5fa222-f6d2-4a77-9cc9-8a70bd322c95-maas-api-xks-ca" pod "dc5fa222-f6d2-4a77-9cc9-8a70bd322c95" (UID: "dc5fa222-f6d2-4a77-9cc9-8a70bd322c95"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:36:53.448142 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.448118    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c8b8d0ba-n2hgt\" (UID: \"97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c8b8d0ba-n2hgt"
Oct 30 19:36:53.448411 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.448396    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/04d55fb9-1145-4041-bb56-e7b398104a74-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8e42d71a-q6p8w\" (UID: \"04d55fb9-1145-4041-bb56-e7b398104a74\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8e42d71a-q6p8w"
Oct 30 19:36:53.474611 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.474585    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t68bp\" (UniqueName: \"kubernetes.io/projected/97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5-kube-api-access-t68bp\") pod \"maas-api-revoke-sub-c8b8d0ba-n2hgt\" (UID: \"97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c8b8d0ba-n2hgt"
Oct 30 19:36:53.476834 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.476815    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-92k7m\" (UniqueName: \"kubernetes.io/projected/04d55fb9-1145-4041-bb56-e7b398104a74-kube-api-access-92k7m\") pod \"maas-api-revoke-sub-8e42d71a-q6p8w\" (UID: \"04d55fb9-1145-4041-bb56-e7b398104a74\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8e42d71a-q6p8w"
Oct 30 19:36:53.547566 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.547534    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hj5pq\" (UniqueName: \"kubernetes.io/projected/dc5fa222-f6d2-4a77-9cc9-8a70bd322c95-kube-api-access-hj5pq\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:53.547566 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.547563    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dc5fa222-f6d2-4a77-9cc9-8a70bd322c95-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:53.564075 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.564051    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c8b8d0ba-n2hgt"
Oct 30 19:36:53.587795 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.587770    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8e42d71a-q6p8w"
Oct 30 19:36:53.876970 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:36:53.876935    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97e12df4_b6c1_4df4_bbd9_cdebfc4c57e5.slice/crio-a3cdeaec846c4329fa47d74620491f8e34fbc51392b728fffc1b434256f45049 WatchSource:0}: Error finding container a3cdeaec846c4329fa47d74620491f8e34fbc51392b728fffc1b434256f45049: Status 404 returned error can't find the container with id a3cdeaec846c4329fa47d74620491f8e34fbc51392b728fffc1b434256f45049
Oct 30 19:36:53.879189 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.879116    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c8b8d0ba-n2hgt"]
Oct 30 19:36:53.916507 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:36:53.916473    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04d55fb9_1145_4041_bb56_e7b398104a74.slice/crio-da6882826e24a3ac4812b2308536156245f9fa593d3b6a87f7695de6dd5f1ca4 WatchSource:0}: Error finding container da6882826e24a3ac4812b2308536156245f9fa593d3b6a87f7695de6dd5f1ca4: Status 404 returned error can't find the container with id da6882826e24a3ac4812b2308536156245f9fa593d3b6a87f7695de6dd5f1ca4
Oct 30 19:36:53.916623 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:53.916549    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8e42d71a-q6p8w"]
Oct 30 19:36:54.162572 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:54.162540    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c8b8d0ba-n2hgt" event={"ID":"97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5","Type":"ContainerStarted","Data":"a3cdeaec846c4329fa47d74620491f8e34fbc51392b728fffc1b434256f45049"}
Oct 30 19:36:54.164179 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:54.164149    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8e42d71a-q6p8w" event={"ID":"04d55fb9-1145-4041-bb56-e7b398104a74","Type":"ContainerStarted","Data":"da6882826e24a3ac4812b2308536156245f9fa593d3b6a87f7695de6dd5f1ca4"}
Oct 30 19:36:54.165546 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:54.165521    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-95a4820f-ng97l"
Oct 30 19:36:54.165546 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:54.165536    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-95a4820f-ng97l" event={"ID":"dc5fa222-f6d2-4a77-9cc9-8a70bd322c95","Type":"ContainerDied","Data":"a5d1618aa39d256124bb306e08caf542bfbbb52279d4d00a4dec0ae4183bf630"}
Oct 30 19:36:54.165676 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:54.165581    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a5d1618aa39d256124bb306e08caf542bfbbb52279d4d00a4dec0ae4183bf630"
Oct 30 19:36:54.971841 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:54.971812    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e78dc66-ssgxc"]
Oct 30 19:36:54.972646 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:54.972630    5907 cpu_manager.go:414] "containerMap: removing container" podUID="dc5fa222-f6d2-4a77-9cc9-8a70bd322c95" containerName="revoke-keys"
Oct 30 19:36:54.972708 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:54.972648    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc5fa222-f6d2-4a77-9cc9-8a70bd322c95" containerName="revoke-keys"
Oct 30 19:36:54.972806 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:54.972797    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc5fa222-f6d2-4a77-9cc9-8a70bd322c95" containerName="revoke-keys"
Oct 30 19:36:54.976765 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:54.976750    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e78dc66-ssgxc"
Oct 30 19:36:54.998308 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:54.998280    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e78dc66-ssgxc"]
Oct 30 19:36:55.091079 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:55.091050    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jhsl\" (UniqueName: \"kubernetes.io/projected/bf7311e4-19fe-4dff-ba2f-e5c310c89c71-kube-api-access-7jhsl\") pod \"maas-api-revoke-sub-2e78dc66-ssgxc\" (UID: \"bf7311e4-19fe-4dff-ba2f-e5c310c89c71\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e78dc66-ssgxc"
Oct 30 19:36:55.091252 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:55.091233    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bf7311e4-19fe-4dff-ba2f-e5c310c89c71-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2e78dc66-ssgxc\" (UID: \"bf7311e4-19fe-4dff-ba2f-e5c310c89c71\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e78dc66-ssgxc"
Oct 30 19:36:55.091382 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:55.091363    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bf7311e4-19fe-4dff-ba2f-e5c310c89c71-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2e78dc66-ssgxc\" (UID: \"bf7311e4-19fe-4dff-ba2f-e5c310c89c71\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e78dc66-ssgxc"
Oct 30 19:36:55.175518 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:55.175485    5907 generic.go:356] "Generic (PLEG): container finished" podID="04d55fb9-1145-4041-bb56-e7b398104a74" containerID="c9ac9cff65596d02f3dbe7eddaa21b1225ccccd830c8eb4ff8ce678012a57032" exitCode=0
Oct 30 19:36:55.175692 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:55.175580    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8e42d71a-q6p8w" event={"ID":"04d55fb9-1145-4041-bb56-e7b398104a74","Type":"ContainerDied","Data":"c9ac9cff65596d02f3dbe7eddaa21b1225ccccd830c8eb4ff8ce678012a57032"}
Oct 30 19:36:55.177459 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:55.177434    5907 generic.go:356] "Generic (PLEG): container finished" podID="97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5" containerID="a8a36f569ab54a83a98e5e6a42263cd1534106d3f67e380d04e0e00db366984d" exitCode=0
Oct 30 19:36:55.177587 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:55.177519    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c8b8d0ba-n2hgt" event={"ID":"97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5","Type":"ContainerDied","Data":"a8a36f569ab54a83a98e5e6a42263cd1534106d3f67e380d04e0e00db366984d"}
Oct 30 19:36:55.192890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:55.192801    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jhsl\" (UniqueName: \"kubernetes.io/projected/bf7311e4-19fe-4dff-ba2f-e5c310c89c71-kube-api-access-7jhsl\") pod \"maas-api-revoke-sub-2e78dc66-ssgxc\" (UID: \"bf7311e4-19fe-4dff-ba2f-e5c310c89c71\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e78dc66-ssgxc"
Oct 30 19:36:55.193003 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:55.192962    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bf7311e4-19fe-4dff-ba2f-e5c310c89c71-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2e78dc66-ssgxc\" (UID: \"bf7311e4-19fe-4dff-ba2f-e5c310c89c71\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e78dc66-ssgxc"
Oct 30 19:36:55.193062 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:55.193047    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bf7311e4-19fe-4dff-ba2f-e5c310c89c71-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2e78dc66-ssgxc\" (UID: \"bf7311e4-19fe-4dff-ba2f-e5c310c89c71\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e78dc66-ssgxc"
Oct 30 19:36:55.193744 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:55.193716    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bf7311e4-19fe-4dff-ba2f-e5c310c89c71-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2e78dc66-ssgxc\" (UID: \"bf7311e4-19fe-4dff-ba2f-e5c310c89c71\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e78dc66-ssgxc"
Oct 30 19:36:55.195586 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:55.195567    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bf7311e4-19fe-4dff-ba2f-e5c310c89c71-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2e78dc66-ssgxc\" (UID: \"bf7311e4-19fe-4dff-ba2f-e5c310c89c71\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e78dc66-ssgxc"
Oct 30 19:36:55.213460 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:55.213434    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jhsl\" (UniqueName: \"kubernetes.io/projected/bf7311e4-19fe-4dff-ba2f-e5c310c89c71-kube-api-access-7jhsl\") pod \"maas-api-revoke-sub-2e78dc66-ssgxc\" (UID: \"bf7311e4-19fe-4dff-ba2f-e5c310c89c71\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e78dc66-ssgxc"
Oct 30 19:36:55.383294 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:55.383259    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e78dc66-ssgxc"
Oct 30 19:36:55.700210 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:36:55.700176    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf7311e4_19fe_4dff_ba2f_e5c310c89c71.slice/crio-3ce38b0f46a605c52cd41e48422d808ae0ceee1be518ff5d40ed058210a3cd62 WatchSource:0}: Error finding container 3ce38b0f46a605c52cd41e48422d808ae0ceee1be518ff5d40ed058210a3cd62: Status 404 returned error can't find the container with id 3ce38b0f46a605c52cd41e48422d808ae0ceee1be518ff5d40ed058210a3cd62
Oct 30 19:36:55.712013 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:55.704181    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e78dc66-ssgxc"]
Oct 30 19:36:56.187572 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:56.187521    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e78dc66-ssgxc" event={"ID":"bf7311e4-19fe-4dff-ba2f-e5c310c89c71","Type":"ContainerStarted","Data":"3ce38b0f46a605c52cd41e48422d808ae0ceee1be518ff5d40ed058210a3cd62"}
Oct 30 19:36:56.363778 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:56.363750    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8e42d71a-q6p8w"
Oct 30 19:36:56.395200 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:56.395175    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c8b8d0ba-n2hgt"
Oct 30 19:36:56.521434 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:56.521395    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5-maas-api-xks-ca\") pod \"97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5\" (UID: \"97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5\") "
Oct 30 19:36:56.521634 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:56.521618    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/04d55fb9-1145-4041-bb56-e7b398104a74-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/04d55fb9-1145-4041-bb56-e7b398104a74-maas-api-openshift-ca\") pod \"04d55fb9-1145-4041-bb56-e7b398104a74\" (UID: \"04d55fb9-1145-4041-bb56-e7b398104a74\") "
Oct 30 19:36:56.521714 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:56.521696    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/04d55fb9-1145-4041-bb56-e7b398104a74-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/04d55fb9-1145-4041-bb56-e7b398104a74-maas-api-xks-ca\") pod \"04d55fb9-1145-4041-bb56-e7b398104a74\" (UID: \"04d55fb9-1145-4041-bb56-e7b398104a74\") "
Oct 30 19:36:56.521955 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:56.521936    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5-maas-api-openshift-ca\") pod \"97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5\" (UID: \"97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5\") "
Oct 30 19:36:56.522081 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:56.522031    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5-kube-api-access-t68bp\" (UniqueName: \"kubernetes.io/projected/97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5-kube-api-access-t68bp\") pod \"97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5\" (UID: \"97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5\") "
Oct 30 19:36:56.522151 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:56.522070    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04d55fb9-1145-4041-bb56-e7b398104a74-maas-api-openshift-ca" pod "04d55fb9-1145-4041-bb56-e7b398104a74" (UID: "04d55fb9-1145-4041-bb56-e7b398104a74"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:36:56.522313 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:56.522290    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/04d55fb9-1145-4041-bb56-e7b398104a74-kube-api-access-92k7m\" (UniqueName: \"kubernetes.io/projected/04d55fb9-1145-4041-bb56-e7b398104a74-kube-api-access-92k7m\") pod \"04d55fb9-1145-4041-bb56-e7b398104a74\" (UID: \"04d55fb9-1145-4041-bb56-e7b398104a74\") "
Oct 30 19:36:56.522469 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:56.522417    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5-maas-api-openshift-ca" pod "97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5" (UID: "97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:36:56.523400 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:56.523359    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/04d55fb9-1145-4041-bb56-e7b398104a74-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:56.523400 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:56.523394    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:56.523816 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:56.523791    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5-maas-api-xks-ca" pod "97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5" (UID: "97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:36:56.524393 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:56.524375    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04d55fb9-1145-4041-bb56-e7b398104a74-maas-api-xks-ca" pod "04d55fb9-1145-4041-bb56-e7b398104a74" (UID: "04d55fb9-1145-4041-bb56-e7b398104a74"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:36:56.524708 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:56.524690    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5-kube-api-access-t68bp" pod "97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5" (UID: "97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5"). InnerVolumeSpecName "kube-api-access-t68bp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:36:56.524926 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:56.524904    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04d55fb9-1145-4041-bb56-e7b398104a74-kube-api-access-92k7m" pod "04d55fb9-1145-4041-bb56-e7b398104a74" (UID: "04d55fb9-1145-4041-bb56-e7b398104a74"). InnerVolumeSpecName "kube-api-access-92k7m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:36:56.624815 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:56.624785    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t68bp\" (UniqueName: \"kubernetes.io/projected/97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5-kube-api-access-t68bp\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:56.624815 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:56.624811    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92k7m\" (UniqueName: \"kubernetes.io/projected/04d55fb9-1145-4041-bb56-e7b398104a74-kube-api-access-92k7m\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:56.624815 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:56.624822    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:56.625058 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:56.624832    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/04d55fb9-1145-4041-bb56-e7b398104a74-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:57.198182 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:57.198148    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c8b8d0ba-n2hgt"
Oct 30 19:36:57.198635 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:57.198188    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c8b8d0ba-n2hgt" event={"ID":"97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5","Type":"ContainerDied","Data":"a3cdeaec846c4329fa47d74620491f8e34fbc51392b728fffc1b434256f45049"}
Oct 30 19:36:57.198635 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:57.198241    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a3cdeaec846c4329fa47d74620491f8e34fbc51392b728fffc1b434256f45049"
Oct 30 19:36:57.200165 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:57.200132    5907 generic.go:356] "Generic (PLEG): container finished" podID="bf7311e4-19fe-4dff-ba2f-e5c310c89c71" containerID="76aa4885a65c7ed3acaed85656f95ac1402fd00135cb8b87caadbd5940167003" exitCode=0
Oct 30 19:36:57.200316 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:57.200235    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e78dc66-ssgxc" event={"ID":"bf7311e4-19fe-4dff-ba2f-e5c310c89c71","Type":"ContainerDied","Data":"76aa4885a65c7ed3acaed85656f95ac1402fd00135cb8b87caadbd5940167003"}
Oct 30 19:36:57.201941 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:57.201915    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8e42d71a-q6p8w" event={"ID":"04d55fb9-1145-4041-bb56-e7b398104a74","Type":"ContainerDied","Data":"da6882826e24a3ac4812b2308536156245f9fa593d3b6a87f7695de6dd5f1ca4"}
Oct 30 19:36:57.202033 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:57.201959    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="da6882826e24a3ac4812b2308536156245f9fa593d3b6a87f7695de6dd5f1ca4"
Oct 30 19:36:57.202033 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:57.201991    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8e42d71a-q6p8w"
Oct 30 19:36:57.446199 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:57.446163    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4f6a322d-cs2r8"]
Oct 30 19:36:57.446916 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:57.446898    5907 cpu_manager.go:414] "containerMap: removing container" podUID="97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5" containerName="revoke-keys"
Oct 30 19:36:57.446971 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:57.446919    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5" containerName="revoke-keys"
Oct 30 19:36:57.446971 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:57.446944    5907 cpu_manager.go:414] "containerMap: removing container" podUID="04d55fb9-1145-4041-bb56-e7b398104a74" containerName="revoke-keys"
Oct 30 19:36:57.446971 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:57.446952    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="04d55fb9-1145-4041-bb56-e7b398104a74" containerName="revoke-keys"
Oct 30 19:36:57.447175 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:57.447163    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5" containerName="revoke-keys"
Oct 30 19:36:57.447209 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:57.447179    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="04d55fb9-1145-4041-bb56-e7b398104a74" containerName="revoke-keys"
Oct 30 19:36:57.451575 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:57.451491    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4f6a322d-cs2r8"
Oct 30 19:36:57.474327 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:57.474286    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4f6a322d-cs2r8"]
Oct 30 19:36:57.542846 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:57.542810    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxgz6\" (UniqueName: \"kubernetes.io/projected/588e2d37-9738-4f5f-ac0e-f4992ae7eb53-kube-api-access-kxgz6\") pod \"maas-api-revoke-sub-4f6a322d-cs2r8\" (UID: \"588e2d37-9738-4f5f-ac0e-f4992ae7eb53\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4f6a322d-cs2r8"
Oct 30 19:36:57.542846 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:57.542894    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/588e2d37-9738-4f5f-ac0e-f4992ae7eb53-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4f6a322d-cs2r8\" (UID: \"588e2d37-9738-4f5f-ac0e-f4992ae7eb53\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4f6a322d-cs2r8"
Oct 30 19:36:57.543146 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:57.542987    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/588e2d37-9738-4f5f-ac0e-f4992ae7eb53-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4f6a322d-cs2r8\" (UID: \"588e2d37-9738-4f5f-ac0e-f4992ae7eb53\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4f6a322d-cs2r8"
Oct 30 19:36:57.644578 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:57.644546    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxgz6\" (UniqueName: \"kubernetes.io/projected/588e2d37-9738-4f5f-ac0e-f4992ae7eb53-kube-api-access-kxgz6\") pod \"maas-api-revoke-sub-4f6a322d-cs2r8\" (UID: \"588e2d37-9738-4f5f-ac0e-f4992ae7eb53\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4f6a322d-cs2r8"
Oct 30 19:36:57.644759 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:57.644614    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/588e2d37-9738-4f5f-ac0e-f4992ae7eb53-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4f6a322d-cs2r8\" (UID: \"588e2d37-9738-4f5f-ac0e-f4992ae7eb53\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4f6a322d-cs2r8"
Oct 30 19:36:57.644759 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:57.644666    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/588e2d37-9738-4f5f-ac0e-f4992ae7eb53-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4f6a322d-cs2r8\" (UID: \"588e2d37-9738-4f5f-ac0e-f4992ae7eb53\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4f6a322d-cs2r8"
Oct 30 19:36:57.645281 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:57.645262    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/588e2d37-9738-4f5f-ac0e-f4992ae7eb53-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4f6a322d-cs2r8\" (UID: \"588e2d37-9738-4f5f-ac0e-f4992ae7eb53\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4f6a322d-cs2r8"
Oct 30 19:36:57.647231 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:57.647205    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/588e2d37-9738-4f5f-ac0e-f4992ae7eb53-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4f6a322d-cs2r8\" (UID: \"588e2d37-9738-4f5f-ac0e-f4992ae7eb53\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4f6a322d-cs2r8"
Oct 30 19:36:57.667210 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:57.667178    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxgz6\" (UniqueName: \"kubernetes.io/projected/588e2d37-9738-4f5f-ac0e-f4992ae7eb53-kube-api-access-kxgz6\") pod \"maas-api-revoke-sub-4f6a322d-cs2r8\" (UID: \"588e2d37-9738-4f5f-ac0e-f4992ae7eb53\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4f6a322d-cs2r8"
Oct 30 19:36:57.767066 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:57.767002    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4f6a322d-cs2r8"
Oct 30 19:36:58.078123 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:36:58.078092    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod588e2d37_9738_4f5f_ac0e_f4992ae7eb53.slice/crio-3f6181e9b88f08629e570495e668ea8755e547d57c734e7ce7e6c6d11db3556a WatchSource:0}: Error finding container 3f6181e9b88f08629e570495e668ea8755e547d57c734e7ce7e6c6d11db3556a: Status 404 returned error can't find the container with id 3f6181e9b88f08629e570495e668ea8755e547d57c734e7ce7e6c6d11db3556a
Oct 30 19:36:58.091881 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:58.081502    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4f6a322d-cs2r8"]
Oct 30 19:36:58.213248 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:58.213214    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4f6a322d-cs2r8" event={"ID":"588e2d37-9738-4f5f-ac0e-f4992ae7eb53","Type":"ContainerStarted","Data":"3f6181e9b88f08629e570495e668ea8755e547d57c734e7ce7e6c6d11db3556a"}
Oct 30 19:36:58.460015 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:58.459993    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e78dc66-ssgxc"
Oct 30 19:36:58.557384 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:58.557350    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bf7311e4-19fe-4dff-ba2f-e5c310c89c71-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bf7311e4-19fe-4dff-ba2f-e5c310c89c71-maas-api-xks-ca\") pod \"bf7311e4-19fe-4dff-ba2f-e5c310c89c71\" (UID: \"bf7311e4-19fe-4dff-ba2f-e5c310c89c71\") "
Oct 30 19:36:58.557574 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:58.557462    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bf7311e4-19fe-4dff-ba2f-e5c310c89c71-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bf7311e4-19fe-4dff-ba2f-e5c310c89c71-maas-api-openshift-ca\") pod \"bf7311e4-19fe-4dff-ba2f-e5c310c89c71\" (UID: \"bf7311e4-19fe-4dff-ba2f-e5c310c89c71\") "
Oct 30 19:36:58.557574 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:58.557525    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bf7311e4-19fe-4dff-ba2f-e5c310c89c71-kube-api-access-7jhsl\" (UniqueName: \"kubernetes.io/projected/bf7311e4-19fe-4dff-ba2f-e5c310c89c71-kube-api-access-7jhsl\") pod \"bf7311e4-19fe-4dff-ba2f-e5c310c89c71\" (UID: \"bf7311e4-19fe-4dff-ba2f-e5c310c89c71\") "
Oct 30 19:36:58.558014 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:58.557986    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf7311e4-19fe-4dff-ba2f-e5c310c89c71-maas-api-openshift-ca" pod "bf7311e4-19fe-4dff-ba2f-e5c310c89c71" (UID: "bf7311e4-19fe-4dff-ba2f-e5c310c89c71"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:36:58.558374 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:58.558342    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bf7311e4-19fe-4dff-ba2f-e5c310c89c71-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:58.560677 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:58.560639    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf7311e4-19fe-4dff-ba2f-e5c310c89c71-kube-api-access-7jhsl" pod "bf7311e4-19fe-4dff-ba2f-e5c310c89c71" (UID: "bf7311e4-19fe-4dff-ba2f-e5c310c89c71"). InnerVolumeSpecName "kube-api-access-7jhsl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:36:58.561443 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:58.561423    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf7311e4-19fe-4dff-ba2f-e5c310c89c71-maas-api-xks-ca" pod "bf7311e4-19fe-4dff-ba2f-e5c310c89c71" (UID: "bf7311e4-19fe-4dff-ba2f-e5c310c89c71"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:36:58.660359 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:58.660332    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bf7311e4-19fe-4dff-ba2f-e5c310c89c71-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:58.660495 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:58.660363    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jhsl\" (UniqueName: \"kubernetes.io/projected/bf7311e4-19fe-4dff-ba2f-e5c310c89c71-kube-api-access-7jhsl\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:36:59.225813 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:59.225784    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e78dc66-ssgxc"
Oct 30 19:36:59.226314 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:59.225816    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e78dc66-ssgxc" event={"ID":"bf7311e4-19fe-4dff-ba2f-e5c310c89c71","Type":"ContainerDied","Data":"3ce38b0f46a605c52cd41e48422d808ae0ceee1be518ff5d40ed058210a3cd62"}
Oct 30 19:36:59.226465 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:59.226441    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3ce38b0f46a605c52cd41e48422d808ae0ceee1be518ff5d40ed058210a3cd62"
Oct 30 19:36:59.227996 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:59.227965    5907 generic.go:356] "Generic (PLEG): container finished" podID="588e2d37-9738-4f5f-ac0e-f4992ae7eb53" containerID="c2fddce35e7897a9d39d52e98ae474922b7b9c88909421f046298d67ea5c4f5b" exitCode=0
Oct 30 19:36:59.228095 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:36:59.228002    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4f6a322d-cs2r8" event={"ID":"588e2d37-9738-4f5f-ac0e-f4992ae7eb53","Type":"ContainerDied","Data":"c2fddce35e7897a9d39d52e98ae474922b7b9c88909421f046298d67ea5c4f5b"}
Oct 30 19:37:00.494929 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.485912    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4f6a322d-cs2r8"
Oct 30 19:37:00.547019 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.546946    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"]
Oct 30 19:37:00.548182 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.548152    5907 cpu_manager.go:414] "containerMap: removing container" podUID="588e2d37-9738-4f5f-ac0e-f4992ae7eb53" containerName="revoke-keys"
Oct 30 19:37:00.548182 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.548181    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="588e2d37-9738-4f5f-ac0e-f4992ae7eb53" containerName="revoke-keys"
Oct 30 19:37:00.548373 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.548257    5907 cpu_manager.go:414] "containerMap: removing container" podUID="bf7311e4-19fe-4dff-ba2f-e5c310c89c71" containerName="revoke-keys"
Oct 30 19:37:00.548373 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.548267    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bf7311e4-19fe-4dff-ba2f-e5c310c89c71" containerName="revoke-keys"
Oct 30 19:37:00.548487 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.548430    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="588e2d37-9738-4f5f-ac0e-f4992ae7eb53" containerName="revoke-keys"
Oct 30 19:37:00.548487 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.548456    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="bf7311e4-19fe-4dff-ba2f-e5c310c89c71" containerName="revoke-keys"
Oct 30 19:37:00.560457 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.560422    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"
Oct 30 19:37:00.585610 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.585581    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"]
Oct 30 19:37:00.608089 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.608048    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"]
Oct 30 19:37:00.608487 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.608452    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/588e2d37-9738-4f5f-ac0e-f4992ae7eb53-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/588e2d37-9738-4f5f-ac0e-f4992ae7eb53-maas-api-openshift-ca\") pod \"588e2d37-9738-4f5f-ac0e-f4992ae7eb53\" (UID: \"588e2d37-9738-4f5f-ac0e-f4992ae7eb53\") "
Oct 30 19:37:00.608587 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.608567    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/588e2d37-9738-4f5f-ac0e-f4992ae7eb53-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/588e2d37-9738-4f5f-ac0e-f4992ae7eb53-maas-api-xks-ca\") pod \"588e2d37-9738-4f5f-ac0e-f4992ae7eb53\" (UID: \"588e2d37-9738-4f5f-ac0e-f4992ae7eb53\") "
Oct 30 19:37:00.608673 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.608656    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/588e2d37-9738-4f5f-ac0e-f4992ae7eb53-kube-api-access-kxgz6\" (UniqueName: \"kubernetes.io/projected/588e2d37-9738-4f5f-ac0e-f4992ae7eb53-kube-api-access-kxgz6\") pod \"588e2d37-9738-4f5f-ac0e-f4992ae7eb53\" (UID: \"588e2d37-9738-4f5f-ac0e-f4992ae7eb53\") "
Oct 30 19:37:00.610774 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.610742    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/588e2d37-9738-4f5f-ac0e-f4992ae7eb53-maas-api-openshift-ca" pod "588e2d37-9738-4f5f-ac0e-f4992ae7eb53" (UID: "588e2d37-9738-4f5f-ac0e-f4992ae7eb53"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:37:00.611054 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.611006    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/685bf818-6ca9-4903-a55e-b4255a635543-istiod-ca-cert\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"
Oct 30 19:37:00.611174 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.611153    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/685bf818-6ca9-4903-a55e-b4255a635543-workload-socket\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"
Oct 30 19:37:00.611293 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.611273    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/685bf818-6ca9-4903-a55e-b4255a635543-istio-data\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"
Oct 30 19:37:00.611414 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.611396    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/685bf818-6ca9-4903-a55e-b4255a635543-workload-certs\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"
Oct 30 19:37:00.611531 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.611461    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/685bf818-6ca9-4903-a55e-b4255a635543-istio-podinfo\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"
Oct 30 19:37:00.611531 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.611498    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97hqk\" (UniqueName: \"kubernetes.io/projected/685bf818-6ca9-4903-a55e-b4255a635543-kube-api-access-97hqk\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"
Oct 30 19:37:00.611657 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.611555    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/685bf818-6ca9-4903-a55e-b4255a635543-istio-token\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"
Oct 30 19:37:00.611657 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.611624    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/685bf818-6ca9-4903-a55e-b4255a635543-credential-socket\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"
Oct 30 19:37:00.611769 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.611662    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/685bf818-6ca9-4903-a55e-b4255a635543-istio-envoy\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"
Oct 30 19:37:00.611892 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.611851    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/588e2d37-9738-4f5f-ac0e-f4992ae7eb53-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:37:00.642390 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.642358    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/588e2d37-9738-4f5f-ac0e-f4992ae7eb53-maas-api-xks-ca" pod "588e2d37-9738-4f5f-ac0e-f4992ae7eb53" (UID: "588e2d37-9738-4f5f-ac0e-f4992ae7eb53"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:37:00.643076 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.642992    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/588e2d37-9738-4f5f-ac0e-f4992ae7eb53-kube-api-access-kxgz6" pod "588e2d37-9738-4f5f-ac0e-f4992ae7eb53" (UID: "588e2d37-9738-4f5f-ac0e-f4992ae7eb53"). InnerVolumeSpecName "kube-api-access-kxgz6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:37:00.656900 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.649113    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"]
Oct 30 19:37:00.656900 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.649238    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"
Oct 30 19:37:00.713575 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.713546    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f23e39e-699c-43ea-bf10-9553aee30c15-workload-certs\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"
Oct 30 19:37:00.713734 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.713584    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/685bf818-6ca9-4903-a55e-b4255a635543-credential-socket\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"
Oct 30 19:37:00.713734 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.713606    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/685bf818-6ca9-4903-a55e-b4255a635543-istio-envoy\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"
Oct 30 19:37:00.713734 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.713716    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f23e39e-699c-43ea-bf10-9553aee30c15-istio-envoy\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"
Oct 30 19:37:00.713834 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.713777    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/685bf818-6ca9-4903-a55e-b4255a635543-istiod-ca-cert\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"
Oct 30 19:37:00.713911 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.713839    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f23e39e-699c-43ea-bf10-9553aee30c15-istio-data\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"
Oct 30 19:37:00.713911 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.713886    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f23e39e-699c-43ea-bf10-9553aee30c15-istio-token\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"
Oct 30 19:37:00.713911 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.713908    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f23e39e-699c-43ea-bf10-9553aee30c15-istiod-ca-cert\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"
Oct 30 19:37:00.714045 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.714028    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/685bf818-6ca9-4903-a55e-b4255a635543-workload-socket\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"
Oct 30 19:37:00.714086 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.714060    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghhbh\" (UniqueName: \"kubernetes.io/projected/0f23e39e-699c-43ea-bf10-9553aee30c15-kube-api-access-ghhbh\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"
Oct 30 19:37:00.714132 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.714099    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f23e39e-699c-43ea-bf10-9553aee30c15-workload-socket\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"
Oct 30 19:37:00.714193 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.714166    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/685bf818-6ca9-4903-a55e-b4255a635543-istio-data\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"
Oct 30 19:37:00.714248 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.714185    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/685bf818-6ca9-4903-a55e-b4255a635543-credential-socket\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"
Oct 30 19:37:00.714308 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.714267    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f23e39e-699c-43ea-bf10-9553aee30c15-credential-socket\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"
Oct 30 19:37:00.714370 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.714339    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/685bf818-6ca9-4903-a55e-b4255a635543-workload-certs\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"
Oct 30 19:37:00.714429 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.714386    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/685bf818-6ca9-4903-a55e-b4255a635543-istio-podinfo\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"
Oct 30 19:37:00.714429 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.714420    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97hqk\" (UniqueName: \"kubernetes.io/projected/685bf818-6ca9-4903-a55e-b4255a635543-kube-api-access-97hqk\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"
Oct 30 19:37:00.714527 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.714454    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f23e39e-699c-43ea-bf10-9553aee30c15-istio-podinfo\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"
Oct 30 19:37:00.714527 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.714487    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/685bf818-6ca9-4903-a55e-b4255a635543-istio-data\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"
Oct 30 19:37:00.714527 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.714497    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/685bf818-6ca9-4903-a55e-b4255a635543-istio-token\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"
Oct 30 19:37:00.714527 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.714452    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/685bf818-6ca9-4903-a55e-b4255a635543-workload-socket\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"
Oct 30 19:37:00.714724 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.714534    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/685bf818-6ca9-4903-a55e-b4255a635543-istiod-ca-cert\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"
Oct 30 19:37:00.714724 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.714652    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/685bf818-6ca9-4903-a55e-b4255a635543-workload-certs\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"
Oct 30 19:37:00.714828 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.714771    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/588e2d37-9738-4f5f-ac0e-f4992ae7eb53-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:37:00.714828 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.714792    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxgz6\" (UniqueName: \"kubernetes.io/projected/588e2d37-9738-4f5f-ac0e-f4992ae7eb53-kube-api-access-kxgz6\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:37:00.716178 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.716157    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/685bf818-6ca9-4903-a55e-b4255a635543-istio-envoy\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"
Oct 30 19:37:00.716796 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.716779    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/685bf818-6ca9-4903-a55e-b4255a635543-istio-podinfo\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"
Oct 30 19:37:00.735310 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.735283    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/685bf818-6ca9-4903-a55e-b4255a635543-istio-token\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"
Oct 30 19:37:00.735403 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.735390    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97hqk\" (UniqueName: \"kubernetes.io/projected/685bf818-6ca9-4903-a55e-b4255a635543-kube-api-access-97hqk\") pod \"e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") " pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"
Oct 30 19:37:00.816679 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.816588    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f23e39e-699c-43ea-bf10-9553aee30c15-workload-certs\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"
Oct 30 19:37:00.816827 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.816706    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f23e39e-699c-43ea-bf10-9553aee30c15-istio-envoy\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"
Oct 30 19:37:00.816827 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.816765    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f23e39e-699c-43ea-bf10-9553aee30c15-istio-data\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"
Oct 30 19:37:00.816827 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.816793    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f23e39e-699c-43ea-bf10-9553aee30c15-istio-token\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"
Oct 30 19:37:00.816827 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.816823    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f23e39e-699c-43ea-bf10-9553aee30c15-istiod-ca-cert\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"
Oct 30 19:37:00.817076 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.816921    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghhbh\" (UniqueName: \"kubernetes.io/projected/0f23e39e-699c-43ea-bf10-9553aee30c15-kube-api-access-ghhbh\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"
Oct 30 19:37:00.817076 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.816983    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f23e39e-699c-43ea-bf10-9553aee30c15-workload-socket\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"
Oct 30 19:37:00.817076 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.817059    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f23e39e-699c-43ea-bf10-9553aee30c15-credential-socket\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"
Oct 30 19:37:00.817222 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.817089    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f23e39e-699c-43ea-bf10-9553aee30c15-workload-certs\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"
Oct 30 19:37:00.817282 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.817217    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f23e39e-699c-43ea-bf10-9553aee30c15-istio-podinfo\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"
Oct 30 19:37:00.817373 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.817350    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f23e39e-699c-43ea-bf10-9553aee30c15-workload-socket\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"
Oct 30 19:37:00.817470 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.817415    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f23e39e-699c-43ea-bf10-9553aee30c15-credential-socket\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"
Oct 30 19:37:00.817519 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.817484    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f23e39e-699c-43ea-bf10-9553aee30c15-istio-data\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"
Oct 30 19:37:00.817844 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.817823    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f23e39e-699c-43ea-bf10-9553aee30c15-istiod-ca-cert\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"
Oct 30 19:37:00.819939 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.819914    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f23e39e-699c-43ea-bf10-9553aee30c15-istio-podinfo\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"
Oct 30 19:37:00.820035 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.819958    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f23e39e-699c-43ea-bf10-9553aee30c15-istio-envoy\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"
Oct 30 19:37:00.837620 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.837597    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f23e39e-699c-43ea-bf10-9553aee30c15-istio-token\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"
Oct 30 19:37:00.837776 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.837758    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghhbh\" (UniqueName: \"kubernetes.io/projected/0f23e39e-699c-43ea-bf10-9553aee30c15-kube-api-access-ghhbh\") pod \"e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") " pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"
Oct 30 19:37:00.893736 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.893710    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"
Oct 30 19:37:00.992888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:00.992175    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"
Oct 30 19:37:01.459068 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:01.454496    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4f6a322d-cs2r8" event={"ID":"588e2d37-9738-4f5f-ac0e-f4992ae7eb53","Type":"ContainerDied","Data":"3f6181e9b88f08629e570495e668ea8755e547d57c734e7ce7e6c6d11db3556a"}
Oct 30 19:37:01.459068 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:01.454558    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3f6181e9b88f08629e570495e668ea8755e547d57c734e7ce7e6c6d11db3556a"
Oct 30 19:37:01.459068 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:01.454630    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4f6a322d-cs2r8"
Oct 30 19:37:02.115287 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:02.013762    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:37:02.115287 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:02.013893    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:37:02.115287 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:02.013947    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:37:02.527010 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:02.526956    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:02.610901 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:02.609516    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"]
Oct 30 19:37:02.976049 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:02.975997    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:02.979914 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:02.979880    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"]
Oct 30 19:37:03.061174 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:03.061126    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:37:03.061418 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:03.061396    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:37:03.061576 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:03.061555    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:37:03.143877 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:03.142075    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv" event={"ID":"685bf818-6ca9-4903-a55e-b4255a635543","Type":"ContainerStarted","Data":"6615373276e95c735ff4bbb56d15b97bd1642588d832fceb46d6a9cce8129fea"}
Oct 30 19:37:03.315940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:03.305016    5907 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:03.542897 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:03.527764    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:03.542897 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:03.528553    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:37:03.736670 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:03.724485    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:04.191932 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:04.151766    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 19:37:04.191932 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:04.152457    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:37:04.193119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:04.192687    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:04.556110 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:04.529947    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:04.882109 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:04.839434    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": context deadline exceeded"
Oct 30 19:37:04.978894 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:04.949226    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Oct 30 19:37:05.186963 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:05.185952    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv" podUID="685bf818-6ca9-4903-a55e-b4255a635543" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Oct 30 19:37:05.186963 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:05.176649    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:05.186963 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:05.130727    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv" event={"ID":"685bf818-6ca9-4903-a55e-b4255a635543","Type":"ContainerStarted","Data":"76727cafec64590f0731e737e2b2d35789ffb5868da2e6f2fef735ca2e6ef694"}
Oct 30 19:37:05.186963 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:05.186331    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"
Oct 30 19:37:05.186963 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:05.186400    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:37:05.186963 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:05.186419    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j" event={"ID":"0f23e39e-699c-43ea-bf10-9553aee30c15","Type":"ContainerStarted","Data":"06731322ab687584e6abc41e6bf8450d2a702884bb8d69e9266631981f61d373"}
Oct 30 19:37:05.186963 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:05.186449    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j" event={"ID":"0f23e39e-699c-43ea-bf10-9553aee30c15","Type":"ContainerStarted","Data":"c62c2dfc5f6f451d04dcf692c1a404c0a8df5824f70c58ce2d8bf502265f188f"}
Oct 30 19:37:05.239097 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:05.235556    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:05.239097 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:05.235907    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:37:05.842465 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:05.842409    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:05.892500 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:05.862313    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": context deadline exceeded"
Oct 30 19:37:06.011894 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:06.010212    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"
Oct 30 19:37:06.070112 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:06.070054    5907 patch_prober.go:29] interesting pod/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:37:06.180943 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:06.174247    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc" podUID="282f4fd8-512c-41b9-9989-0b84f2de1ce5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:06.180943 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:06.137535    5907 patch_prober.go:29] interesting pod/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 19:37:06.180943 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:06.174359    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j" podUID="0f23e39e-699c-43ea-bf10-9553aee30c15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 19:37:06.251585 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:06.251539    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:06.307887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:06.290637    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:06.307887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:06.299708    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"]
Oct 30 19:37:06.408356 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:06.408313    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Oct 30 19:37:06.408658 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:06.408615    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv" podUID="685bf818-6ca9-4903-a55e-b4255a635543" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Oct 30 19:37:06.894038 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:06.893980    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:07.211002 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:07.189919    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Oct 30 19:37:07.211002 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:07.189999    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv" podUID="685bf818-6ca9-4903-a55e-b4255a635543" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Oct 30 19:37:07.211002 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:06.906650    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": context deadline exceeded"
Oct 30 19:37:07.268406 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:07.268359    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:07.353897 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:07.353054    5907 patch_prober.go:29] interesting pod/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 19:37:07.353897 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:07.353141    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j" podUID="0f23e39e-699c-43ea-bf10-9553aee30c15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 19:37:07.353897 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:07.353360    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 19:37:07.593934 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:07.573034    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"
Oct 30 19:37:07.690425 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:07.690373    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-62b91eb5-openshift-default-dockercfg-lt72w\""
Oct 30 19:37:07.975516 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:07.973686    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-wd6m6" podUID="0b52bf40-f75b-465b-8947-10ea73874f85" containerName="main" probeResult="failure" output="Get \"https://10.134.0.37:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:08.083213 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.067437    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-workload-certs\") pod \"e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") " pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"
Oct 30 19:37:08.083213 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.067509    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-istio-data\") pod \"e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") " pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"
Oct 30 19:37:08.083213 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.067541    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nbgt\" (UniqueName: \"kubernetes.io/projected/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-kube-api-access-8nbgt\") pod \"e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") " pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"
Oct 30 19:37:08.083213 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.067595    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-istio-envoy\") pod \"e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") " pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"
Oct 30 19:37:08.083213 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.067721    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-workload-socket\") pod \"e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") " pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"
Oct 30 19:37:08.083213 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.067764    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-istio-podinfo\") pod \"e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") " pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"
Oct 30 19:37:08.083213 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.067837    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-istiod-ca-cert\") pod \"e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") " pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"
Oct 30 19:37:08.083213 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.068030    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-istio-token\") pod \"e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") " pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"
Oct 30 19:37:08.083213 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.068073    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-credential-socket\") pod \"e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") " pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"
Oct 30 19:37:08.171933 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.171838    5907 patch_prober.go:29] interesting pod/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 19:37:08.179052 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.178983    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j" podUID="0f23e39e-699c-43ea-bf10-9553aee30c15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 19:37:08.201098 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.201048    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": context deadline exceeded"
Oct 30 19:37:08.379814 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.364460    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-istio-podinfo\") pod \"e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") " pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"
Oct 30 19:37:08.379814 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.364573    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-istiod-ca-cert\") pod \"e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") " pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"
Oct 30 19:37:08.379814 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.364754    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-istio-token\") pod \"e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") " pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"
Oct 30 19:37:08.379814 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.364803    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-credential-socket\") pod \"e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") " pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"
Oct 30 19:37:08.379814 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.364944    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-workload-certs\") pod \"e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") " pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"
Oct 30 19:37:08.379814 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.365029    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-istio-data\") pod \"e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") " pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"
Oct 30 19:37:08.379814 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.365061    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nbgt\" (UniqueName: \"kubernetes.io/projected/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-kube-api-access-8nbgt\") pod \"e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") " pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"
Oct 30 19:37:08.379814 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.365135    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-istio-envoy\") pod \"e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") " pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"
Oct 30 19:37:08.379814 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.365387    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-workload-socket\") pod \"e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") " pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"
Oct 30 19:37:08.379814 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.366067    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-workload-socket\") pod \"e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") " pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"
Oct 30 19:37:08.598897 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.552349    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:08.598897 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.560243    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:08.598897 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.560942    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:08.659814 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.616319    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Oct 30 19:37:08.659814 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.616385    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv" podUID="685bf818-6ca9-4903-a55e-b4255a635543" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Oct 30 19:37:08.659814 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.644928    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-istio-data\") pod \"e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") " pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"
Oct 30 19:37:08.659814 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.645628    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-istiod-ca-cert\") pod \"e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") " pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"
Oct 30 19:37:08.688121 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.688086    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-credential-socket\") pod \"e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") " pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"
Oct 30 19:37:08.830515 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.712356    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-workload-certs\") pod \"e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") " pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"
Oct 30 19:37:08.859383 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.859319    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv" podStartSLOduration=8.859300717 podStartE2EDuration="8.859300717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:37:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:37:04.694812586 +0000 UTC m=+936.537744528" watchObservedRunningTime="2026-10-30 19:37:08.859300717 +0000 UTC m=+940.702232659"
Oct 30 19:37:08.870899 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.868943    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nbgt\" (UniqueName: \"kubernetes.io/projected/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-kube-api-access-8nbgt\") pod \"e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") " pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"
Oct 30 19:37:08.899277 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.899231    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Oct 30 19:37:08.900070 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.900007    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv" podUID="685bf818-6ca9-4903-a55e-b4255a635543" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Oct 30 19:37:08.935155 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:08.935042    5907 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:09.066890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:09.046049    5907 patch_prober.go:29] interesting pod/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 19:37:09.081957 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:09.081896    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j" podUID="0f23e39e-699c-43ea-bf10-9553aee30c15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 19:37:09.106680 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:09.106643    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-istio-podinfo\") pod \"e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") " pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"
Oct 30 19:37:09.213120 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:09.213031    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-istio-envoy\") pod \"e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") " pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"
Oct 30 19:37:09.226225 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:09.226179    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:37:09.226514 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:09.226477    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc" podUID="e440f6c0-de4c-4a55-af89-7ea0552419a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:09.316887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:09.279774    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": context deadline exceeded"
Oct 30 19:37:09.355891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:09.332914    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j" podStartSLOduration=9.332892587 podStartE2EDuration="9.332892587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:37:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:37:05.408739778 +0000 UTC m=+937.251671718" watchObservedRunningTime="2026-10-30 19:37:09.332892587 +0000 UTC m=+941.175824527"
Oct 30 19:37:09.441932 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:09.398391    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-istio-token\") pod \"e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") " pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"
Oct 30 19:37:09.634904 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:09.462953    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"
Oct 30 19:37:09.635833 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:09.524161    5907 patch_prober.go:29] interesting pod/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:37:09.660995 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:09.660949    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6" podUID="1ed86722-1b31-4348-ac72-136909f507c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:09.661757 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:09.563950    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:09.662243 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:09.563987    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": context deadline exceeded"
Oct 30 19:37:09.673933 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:09.560360    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 19:37:09.700351 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:09.524470    5907 patch_prober.go:29] interesting pod/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:37:09.700667 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:09.700636    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5" podUID="819b9937-3bc3-47ac-9631-b6c4fa30a80d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:09.700986 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:37:09.661675    5907 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.296s"
Oct 30 19:37:09.774919 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:09.745779    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"]
Oct 30 19:37:09.858497 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:09.858441    5907 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:09.962944 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:09.952007    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Oct 30 19:37:09.962944 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:09.952064    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv" podUID="685bf818-6ca9-4903-a55e-b4255a635543" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Oct 30 19:37:10.035100 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:10.034306    5907 patch_prober.go:29] interesting pod/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 19:37:10.035100 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:10.034368    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j" podUID="0f23e39e-699c-43ea-bf10-9553aee30c15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 19:37:10.157408 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:10.157362    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:37:10.158107 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:10.157834    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm" podUID="ce80464b-ca17-4aed-8228-e19549d3e1de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:10.199686 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:10.199578    5907 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:10.309961 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:10.304458    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:10.374097 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:10.374046    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:37:10.389107 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:10.389042    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb" podUID="283578cb-7837-4d4a-b91f-dee1d0d74322" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:10.699613 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:10.683742    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": context deadline exceeded"
Oct 30 19:37:10.746238 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:10.746185    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:10.764889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:10.694644    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 19:37:11.082224 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.082090    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Oct 30 19:37:11.082493 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.082461    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv" podUID="685bf818-6ca9-4903-a55e-b4255a635543" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Oct 30 19:37:11.185891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.180371    5907 patch_prober.go:29] interesting pod/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 19:37:11.185891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.180478    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j" podUID="0f23e39e-699c-43ea-bf10-9553aee30c15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 19:37:11.369366 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:37:11.369303    5907 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.003s"
Oct 30 19:37:11.369736 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.369711    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:37:11.384363 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.384074    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:37:11.386503 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.386475    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:37:11.393174 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.387622    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:37:11.584889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.567329    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"]
Oct 30 19:37:11.605442 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.605411    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:37:11.605627 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.605414    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"]
Oct 30 19:37:11.668747 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.668628    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-5016f41a-gw-openshift-default-dockercfg-lnk5v\""
Oct 30 19:37:11.697313 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.697231    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-istio-podinfo\") pod \"e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") " pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:37:11.697608 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.697574    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-istio-data\") pod \"e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") " pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:37:11.698202 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.698165    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-workload-certs\") pod \"e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") " pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:37:11.698396 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.698371    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-istio-envoy\") pod \"e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") " pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:37:11.698543 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.698526    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-workload-socket\") pod \"e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") " pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:37:11.698876 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.698838    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-credential-socket\") pod \"e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") " pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:37:11.699154 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.699134    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cd6tk\" (UniqueName: \"kubernetes.io/projected/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-kube-api-access-cd6tk\") pod \"e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") " pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:37:11.699321 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.699297    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-istiod-ca-cert\") pod \"e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") " pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:37:11.699433 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.699376    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-istio-token\") pod \"e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") " pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:37:11.801350 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.801310    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cd6tk\" (UniqueName: \"kubernetes.io/projected/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-kube-api-access-cd6tk\") pod \"e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") " pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:37:11.801975 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.801944    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-istiod-ca-cert\") pod \"e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") " pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:37:11.802232 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.802215    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-istio-token\") pod \"e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") " pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:37:11.802380 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.802367    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-istio-podinfo\") pod \"e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") " pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:37:11.802481 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.802469    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-istio-data\") pod \"e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") " pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:37:11.802599 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.802585    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-workload-certs\") pod \"e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") " pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:37:11.802721 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.802705    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-istio-envoy\") pod \"e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") " pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:37:11.802833 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.802820    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-workload-socket\") pod \"e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") " pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:37:11.803777 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.803752    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-credential-socket\") pod \"e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") " pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:37:11.804685 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.804661    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-credential-socket\") pod \"e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") " pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:37:11.805156 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.805134    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-istio-data\") pod \"e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") " pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:37:11.805683 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.805662    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-workload-certs\") pod \"e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") " pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:37:11.884888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.811737    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-workload-socket\") pod \"e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") " pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:37:11.890926 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.851605    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-istiod-ca-cert\") pod \"e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") " pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:37:12.006984 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.894363    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Oct 30 19:37:12.006984 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.894430    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv" podUID="685bf818-6ca9-4903-a55e-b4255a635543" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Oct 30 19:37:12.006984 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:11.934953    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-istio-token\") pod \"e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") " pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:37:12.139553 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:12.139516    5907 patch_prober.go:29] interesting pod/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 19:37:12.139840 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:12.139806    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j" podUID="0f23e39e-699c-43ea-bf10-9553aee30c15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 19:37:12.153741 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:12.153693    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"]
Oct 30 19:37:12.326677 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:12.326592    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cd6tk\" (UniqueName: \"kubernetes.io/projected/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-kube-api-access-cd6tk\") pod \"e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") " pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:37:12.483031 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:12.333569    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-istio-podinfo\") pod \"e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") " pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:37:12.483265 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:12.333775    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-istio-envoy\") pod \"e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") " pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:37:12.483367 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:12.359476    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:37:12.483595 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:12.483562    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:37:12.483763 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:12.483745    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:37:12.508285 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:12.508253    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz" event={"ID":"a28e0e27-b1cb-4d23-a869-a291ccfd30ae","Type":"ContainerStarted","Data":"8c614da420d3dd3d77bd2721fedbcc257f3449a5cfae542ee56bedb218d91d94"}
Oct 30 19:37:12.587239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:12.587207    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:37:12.908247 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:12.908207    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Oct 30 19:37:12.908830 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:12.908799    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv" podUID="685bf818-6ca9-4903-a55e-b4255a635543" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Oct 30 19:37:13.001885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:13.000450    5907 patch_prober.go:29] interesting pod/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 19:37:13.001885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:13.000509    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j" podUID="0f23e39e-699c-43ea-bf10-9553aee30c15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 19:37:13.455321 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:13.367747    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:37:13.455321 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:13.367840    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:37:13.455321 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:13.397516    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:37:13.688193 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:13.688154    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"]
Oct 30 19:37:13.904888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:13.898953    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Oct 30 19:37:13.904888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:13.899013    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv" podUID="685bf818-6ca9-4903-a55e-b4255a635543" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Oct 30 19:37:13.912593 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:13.912501    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh" event={"ID":"ce9284b5-c1c6-4390-a5aa-410c7538e9eb","Type":"ContainerStarted","Data":"c1ab4b7d46d962bd9f4797a2374c3a1b3f724698f7cd6a465ec95db5587dd0d0"}
Oct 30 19:37:13.917816 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:13.917116    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz" event={"ID":"a28e0e27-b1cb-4d23-a869-a291ccfd30ae","Type":"ContainerStarted","Data":"1070687e67289e59a279dbcf53d3a636424a458821396b18052231793cb79091"}
Oct 30 19:37:13.981126 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:13.979721    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz" podStartSLOduration=9.979702164 podStartE2EDuration="9.979702164s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:37:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:37:13.978168695 +0000 UTC m=+945.821100636" watchObservedRunningTime="2026-10-30 19:37:13.979702164 +0000 UTC m=+945.822634103"
Oct 30 19:37:14.003061 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:13.995063    5907 patch_prober.go:29] interesting pod/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 19:37:14.003061 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:13.995139    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j" podUID="0f23e39e-699c-43ea-bf10-9553aee30c15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 19:37:14.472127 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:14.472089    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"
Oct 30 19:37:14.500436 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:14.499925    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"
Oct 30 19:37:14.896450 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:14.894122    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Oct 30 19:37:14.896450 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:14.894181    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv" podUID="685bf818-6ca9-4903-a55e-b4255a635543" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Oct 30 19:37:14.976559 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:14.976520    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh" event={"ID":"ce9284b5-c1c6-4390-a5aa-410c7538e9eb","Type":"ContainerStarted","Data":"42591b20effe399ad712e14ff8d7ce6d7a59128a3ff52a71e3cc807d7904b742"}
Oct 30 19:37:14.977132 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:14.977106    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"
Oct 30 19:37:14.977993 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:14.977964    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"
Oct 30 19:37:14.994137 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:14.994108    5907 patch_prober.go:29] interesting pod/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 19:37:14.994261 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:14.994155    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j" podUID="0f23e39e-699c-43ea-bf10-9553aee30c15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 19:37:15.066213 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:15.066156    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh" podStartSLOduration=4.066137563 podStartE2EDuration="4.066137563s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:37:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:37:15.06280379 +0000 UTC m=+946.905735740" watchObservedRunningTime="2026-10-30 19:37:15.066137563 +0000 UTC m=+946.909069502"
Oct 30 19:37:15.599362 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:15.597895    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:37:15.663716 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:15.663197    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:37:15.894578 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:15.894548    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"
Oct 30 19:37:15.894994 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:15.894968    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Oct 30 19:37:15.895127 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:15.895031    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv" podUID="685bf818-6ca9-4903-a55e-b4255a635543" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Oct 30 19:37:15.992022 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:15.991325    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:37:15.992904 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:15.992615    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:37:15.993651 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:15.993633    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"
Oct 30 19:37:15.993926 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:15.993897    5907 patch_prober.go:29] interesting pod/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 19:37:15.994021 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:15.993958    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j" podUID="0f23e39e-699c-43ea-bf10-9553aee30c15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 19:37:16.895661 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:16.895627    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Oct 30 19:37:16.895844 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:16.895687    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv" podUID="685bf818-6ca9-4903-a55e-b4255a635543" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Oct 30 19:37:16.994724 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:16.994687    5907 patch_prober.go:29] interesting pod/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 19:37:16.995149 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:16.994755    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j" podUID="0f23e39e-699c-43ea-bf10-9553aee30c15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 19:37:17.511386 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:37:17.511328    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:37:17.599158 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:17.599123    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-62b91eb5-6579c7c78c-gtv77"]
Oct 30 19:37:17.605772 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:17.605701    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-62b91eb5-6579c7c78c-gtv77"
Oct 30 19:37:17.622157 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:17.622128    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-62b91eb5-64bb64b574-5rt9l"]
Oct 30 19:37:17.635688 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:17.628785    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-62b91eb5-64bb64b574-5rt9l"
Oct 30 19:37:17.636270 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:17.636203    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-62b91eb5-6579c7c78c-gtv77"]
Oct 30 19:37:17.640346 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:17.638108    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-62b91eb5\""
Oct 30 19:37:17.647527 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:17.646839    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-62b91eb5-dockercfg-bkgpb\""
Oct 30 19:37:17.686876 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:17.685331    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cthl\" (UniqueName: \"kubernetes.io/projected/4edc707a-3196-48f1-919a-98e5b12a234e-kube-api-access-9cthl\") pod \"payload-processing-e2e-ait-62b91eb5-64bb64b574-5rt9l\" (UID: \"4edc707a-3196-48f1-919a-98e5b12a234e\") " pod="openshift-ingress/payload-processing-e2e-ait-62b91eb5-64bb64b574-5rt9l"
Oct 30 19:37:17.686876 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:17.685552    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2645bd0d-99ac-4e8d-a136-f3d37545d517-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-62b91eb5-6579c7c78c-gtv77\" (UID: \"2645bd0d-99ac-4e8d-a136-f3d37545d517\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-62b91eb5-6579c7c78c-gtv77"
Oct 30 19:37:17.686876 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:17.685755    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fk67m\" (UniqueName: \"kubernetes.io/projected/2645bd0d-99ac-4e8d-a136-f3d37545d517-kube-api-access-fk67m\") pod \"payload-pre-processing-e2e-ait-62b91eb5-6579c7c78c-gtv77\" (UID: \"2645bd0d-99ac-4e8d-a136-f3d37545d517\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-62b91eb5-6579c7c78c-gtv77"
Oct 30 19:37:17.686876 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:17.685787    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4edc707a-3196-48f1-919a-98e5b12a234e-plugins-config-volume\") pod \"payload-processing-e2e-ait-62b91eb5-64bb64b574-5rt9l\" (UID: \"4edc707a-3196-48f1-919a-98e5b12a234e\") " pod="openshift-ingress/payload-processing-e2e-ait-62b91eb5-64bb64b574-5rt9l"
Oct 30 19:37:17.697924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:17.697750    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-62b91eb5-64bb64b574-5rt9l"]
Oct 30 19:37:17.826936 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:17.820602    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2645bd0d-99ac-4e8d-a136-f3d37545d517-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-62b91eb5-6579c7c78c-gtv77\" (UID: \"2645bd0d-99ac-4e8d-a136-f3d37545d517\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-62b91eb5-6579c7c78c-gtv77"
Oct 30 19:37:17.826936 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:17.820845    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fk67m\" (UniqueName: \"kubernetes.io/projected/2645bd0d-99ac-4e8d-a136-f3d37545d517-kube-api-access-fk67m\") pod \"payload-pre-processing-e2e-ait-62b91eb5-6579c7c78c-gtv77\" (UID: \"2645bd0d-99ac-4e8d-a136-f3d37545d517\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-62b91eb5-6579c7c78c-gtv77"
Oct 30 19:37:17.826936 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:17.820897    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4edc707a-3196-48f1-919a-98e5b12a234e-plugins-config-volume\") pod \"payload-processing-e2e-ait-62b91eb5-64bb64b574-5rt9l\" (UID: \"4edc707a-3196-48f1-919a-98e5b12a234e\") " pod="openshift-ingress/payload-processing-e2e-ait-62b91eb5-64bb64b574-5rt9l"
Oct 30 19:37:17.826936 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:17.821009    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cthl\" (UniqueName: \"kubernetes.io/projected/4edc707a-3196-48f1-919a-98e5b12a234e-kube-api-access-9cthl\") pod \"payload-processing-e2e-ait-62b91eb5-64bb64b574-5rt9l\" (UID: \"4edc707a-3196-48f1-919a-98e5b12a234e\") " pod="openshift-ingress/payload-processing-e2e-ait-62b91eb5-64bb64b574-5rt9l"
Oct 30 19:37:17.826936 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:17.822545    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2645bd0d-99ac-4e8d-a136-f3d37545d517-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-62b91eb5-6579c7c78c-gtv77\" (UID: \"2645bd0d-99ac-4e8d-a136-f3d37545d517\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-62b91eb5-6579c7c78c-gtv77"
Oct 30 19:37:17.826936 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:17.823585    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4edc707a-3196-48f1-919a-98e5b12a234e-plugins-config-volume\") pod \"payload-processing-e2e-ait-62b91eb5-64bb64b574-5rt9l\" (UID: \"4edc707a-3196-48f1-919a-98e5b12a234e\") " pod="openshift-ingress/payload-processing-e2e-ait-62b91eb5-64bb64b574-5rt9l"
Oct 30 19:37:18.034128 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:18.034078    5907 patch_prober.go:29] interesting pod/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 19:37:18.034652 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:18.034155    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j" podUID="0f23e39e-699c-43ea-bf10-9553aee30c15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 19:37:18.184123 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:18.184068    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cthl\" (UniqueName: \"kubernetes.io/projected/4edc707a-3196-48f1-919a-98e5b12a234e-kube-api-access-9cthl\") pod \"payload-processing-e2e-ait-62b91eb5-64bb64b574-5rt9l\" (UID: \"4edc707a-3196-48f1-919a-98e5b12a234e\") " pod="openshift-ingress/payload-processing-e2e-ait-62b91eb5-64bb64b574-5rt9l"
Oct 30 19:37:18.188219 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:18.188153    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fk67m\" (UniqueName: \"kubernetes.io/projected/2645bd0d-99ac-4e8d-a136-f3d37545d517-kube-api-access-fk67m\") pod \"payload-pre-processing-e2e-ait-62b91eb5-6579c7c78c-gtv77\" (UID: \"2645bd0d-99ac-4e8d-a136-f3d37545d517\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-62b91eb5-6579c7c78c-gtv77"
Oct 30 19:37:18.231272 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:18.230766    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-62b91eb5-6579c7c78c-gtv77"
Oct 30 19:37:18.283313 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:18.266419    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-62b91eb5-64bb64b574-5rt9l"
Oct 30 19:37:18.972189 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:18.968428    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"
Oct 30 19:37:19.006097 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:19.001157    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"
Oct 30 19:37:19.082418 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:19.082393    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-62b91eb5-6579c7c78c-gtv77"]
Oct 30 19:37:19.147284 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:19.147248    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-62b91eb5-64bb64b574-5rt9l"]
Oct 30 19:37:19.198085 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:19.198050    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-62b91eb5-6579c7c78c-gtv77" event={"ID":"2645bd0d-99ac-4e8d-a136-f3d37545d517","Type":"ContainerStarted","Data":"cf097438beaadaaaa361bec5afb5fb86d3a8f41c9ede50aa7f45da64d5d1107c"}
Oct 30 19:37:19.200956 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:19.200933    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-62b91eb5-64bb64b574-5rt9l" event={"ID":"4edc707a-3196-48f1-919a-98e5b12a234e","Type":"ContainerStarted","Data":"229b7e52a920f4842351358ee2d698cd591dd7224be21998bc324e0751830960"}
Oct 30 19:37:19.257149 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:19.257054    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-5016f41a-bff455988-57j6h"]
Oct 30 19:37:19.263843 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:19.263700    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-5016f41a-bff455988-57j6h"
Oct 30 19:37:19.280893 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:19.279383    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-5016f41a\""
Oct 30 19:37:19.291998 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:19.290935    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-5016f41a-dockercfg-6dvp5\""
Oct 30 19:37:19.292557 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:19.292534    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-5016f41a-bff455988-57j6h"]
Oct 30 19:37:19.318274 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:19.318246    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-5016f41a-bd5df848c-mvxr6"]
Oct 30 19:37:19.335785 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:19.334580    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5016f41a-bd5df848c-mvxr6"
Oct 30 19:37:19.342219 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:19.341124    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6flk\" (UniqueName: \"kubernetes.io/projected/a3c0d05d-67f1-4e07-82d0-516ff973b2f2-kube-api-access-h6flk\") pod \"payload-processing-e2e-migrate-5016f41a-bff455988-57j6h\" (UID: \"a3c0d05d-67f1-4e07-82d0-516ff973b2f2\") " pod="openshift-ingress/payload-processing-e2e-migrate-5016f41a-bff455988-57j6h"
Oct 30 19:37:19.342219 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:19.341589    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3c0d05d-67f1-4e07-82d0-516ff973b2f2-plugins-config-volume\") pod \"payload-processing-e2e-migrate-5016f41a-bff455988-57j6h\" (UID: \"a3c0d05d-67f1-4e07-82d0-516ff973b2f2\") " pod="openshift-ingress/payload-processing-e2e-migrate-5016f41a-bff455988-57j6h"
Oct 30 19:37:19.342219 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:19.341745    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdbr5\" (UniqueName: \"kubernetes.io/projected/7659a68f-a908-46c0-a61c-74f827b84339-kube-api-access-cdbr5\") pod \"payload-pre-processing-e2e-migrate-5016f41a-bd5df848c-mvxr6\" (UID: \"7659a68f-a908-46c0-a61c-74f827b84339\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-5016f41a-bd5df848c-mvxr6"
Oct 30 19:37:19.342219 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:19.341805    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7659a68f-a908-46c0-a61c-74f827b84339-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-5016f41a-bd5df848c-mvxr6\" (UID: \"7659a68f-a908-46c0-a61c-74f827b84339\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-5016f41a-bd5df848c-mvxr6"
Oct 30 19:37:19.541940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:19.418176    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-5016f41a-bd5df848c-mvxr6"]
Oct 30 19:37:19.541940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:19.449975    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6flk\" (UniqueName: \"kubernetes.io/projected/a3c0d05d-67f1-4e07-82d0-516ff973b2f2-kube-api-access-h6flk\") pod \"payload-processing-e2e-migrate-5016f41a-bff455988-57j6h\" (UID: \"a3c0d05d-67f1-4e07-82d0-516ff973b2f2\") " pod="openshift-ingress/payload-processing-e2e-migrate-5016f41a-bff455988-57j6h"
Oct 30 19:37:19.541940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:19.450286    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3c0d05d-67f1-4e07-82d0-516ff973b2f2-plugins-config-volume\") pod \"payload-processing-e2e-migrate-5016f41a-bff455988-57j6h\" (UID: \"a3c0d05d-67f1-4e07-82d0-516ff973b2f2\") " pod="openshift-ingress/payload-processing-e2e-migrate-5016f41a-bff455988-57j6h"
Oct 30 19:37:19.541940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:19.450760    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdbr5\" (UniqueName: \"kubernetes.io/projected/7659a68f-a908-46c0-a61c-74f827b84339-kube-api-access-cdbr5\") pod \"payload-pre-processing-e2e-migrate-5016f41a-bd5df848c-mvxr6\" (UID: \"7659a68f-a908-46c0-a61c-74f827b84339\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-5016f41a-bd5df848c-mvxr6"
Oct 30 19:37:19.541940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:19.450934    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7659a68f-a908-46c0-a61c-74f827b84339-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-5016f41a-bd5df848c-mvxr6\" (UID: \"7659a68f-a908-46c0-a61c-74f827b84339\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-5016f41a-bd5df848c-mvxr6"
Oct 30 19:37:19.541940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:19.452470    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3c0d05d-67f1-4e07-82d0-516ff973b2f2-plugins-config-volume\") pod \"payload-processing-e2e-migrate-5016f41a-bff455988-57j6h\" (UID: \"a3c0d05d-67f1-4e07-82d0-516ff973b2f2\") " pod="openshift-ingress/payload-processing-e2e-migrate-5016f41a-bff455988-57j6h"
Oct 30 19:37:19.541940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:19.453206    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7659a68f-a908-46c0-a61c-74f827b84339-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-5016f41a-bd5df848c-mvxr6\" (UID: \"7659a68f-a908-46c0-a61c-74f827b84339\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-5016f41a-bd5df848c-mvxr6"
Oct 30 19:37:19.541940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:19.505492    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6flk\" (UniqueName: \"kubernetes.io/projected/a3c0d05d-67f1-4e07-82d0-516ff973b2f2-kube-api-access-h6flk\") pod \"payload-processing-e2e-migrate-5016f41a-bff455988-57j6h\" (UID: \"a3c0d05d-67f1-4e07-82d0-516ff973b2f2\") " pod="openshift-ingress/payload-processing-e2e-migrate-5016f41a-bff455988-57j6h"
Oct 30 19:37:19.624276 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:19.618566    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-5016f41a-bff455988-57j6h"
Oct 30 19:37:19.723474 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:19.723379    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdbr5\" (UniqueName: \"kubernetes.io/projected/7659a68f-a908-46c0-a61c-74f827b84339-kube-api-access-cdbr5\") pod \"payload-pre-processing-e2e-migrate-5016f41a-bd5df848c-mvxr6\" (UID: \"7659a68f-a908-46c0-a61c-74f827b84339\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-5016f41a-bd5df848c-mvxr6"
Oct 30 19:37:19.738637 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:19.738611    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5016f41a-bd5df848c-mvxr6"
Oct 30 19:37:19.903362 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:19.903304    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e3b90b28-28lz8"]
Oct 30 19:37:19.915061 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:19.915039    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3b90b28-28lz8"
Oct 30 19:37:19.930477 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:19.928332    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e3b90b28-28lz8"]
Oct 30 19:37:19.932912 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:19.932705    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:37:19.932912 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:19.932821    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:37:19.975721 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:19.975686    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1c7e341b-211b-4628-a7f6-e96271655434-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e3b90b28-28lz8\" (UID: \"1c7e341b-211b-4628-a7f6-e96271655434\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3b90b28-28lz8"
Oct 30 19:37:19.976071 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:19.976049    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1c7e341b-211b-4628-a7f6-e96271655434-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e3b90b28-28lz8\" (UID: \"1c7e341b-211b-4628-a7f6-e96271655434\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3b90b28-28lz8"
Oct 30 19:37:19.976188 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:19.976170    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsb7w\" (UniqueName: \"kubernetes.io/projected/1c7e341b-211b-4628-a7f6-e96271655434-kube-api-access-lsb7w\") pod \"maas-api-revoke-sub-e3b90b28-28lz8\" (UID: \"1c7e341b-211b-4628-a7f6-e96271655434\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3b90b28-28lz8"
Oct 30 19:37:20.003112 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:20.003079    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"
Oct 30 19:37:20.005231 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:20.005206    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"
Oct 30 19:37:20.092899 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:20.092793    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1c7e341b-211b-4628-a7f6-e96271655434-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e3b90b28-28lz8\" (UID: \"1c7e341b-211b-4628-a7f6-e96271655434\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3b90b28-28lz8"
Oct 30 19:37:20.093410 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:20.092943    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsb7w\" (UniqueName: \"kubernetes.io/projected/1c7e341b-211b-4628-a7f6-e96271655434-kube-api-access-lsb7w\") pod \"maas-api-revoke-sub-e3b90b28-28lz8\" (UID: \"1c7e341b-211b-4628-a7f6-e96271655434\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3b90b28-28lz8"
Oct 30 19:37:20.093410 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:20.093094    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1c7e341b-211b-4628-a7f6-e96271655434-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e3b90b28-28lz8\" (UID: \"1c7e341b-211b-4628-a7f6-e96271655434\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3b90b28-28lz8"
Oct 30 19:37:20.096373 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:20.096348    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1c7e341b-211b-4628-a7f6-e96271655434-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e3b90b28-28lz8\" (UID: \"1c7e341b-211b-4628-a7f6-e96271655434\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3b90b28-28lz8"
Oct 30 19:37:20.101358 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:20.101336    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1c7e341b-211b-4628-a7f6-e96271655434-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e3b90b28-28lz8\" (UID: \"1c7e341b-211b-4628-a7f6-e96271655434\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3b90b28-28lz8"
Oct 30 19:37:20.116675 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:20.116635    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsb7w\" (UniqueName: \"kubernetes.io/projected/1c7e341b-211b-4628-a7f6-e96271655434-kube-api-access-lsb7w\") pod \"maas-api-revoke-sub-e3b90b28-28lz8\" (UID: \"1c7e341b-211b-4628-a7f6-e96271655434\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3b90b28-28lz8"
Oct 30 19:37:20.214830 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:20.214792    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-62b91eb5-6579c7c78c-gtv77" event={"ID":"2645bd0d-99ac-4e8d-a136-f3d37545d517","Type":"ContainerStarted","Data":"fa3a5cd6e1b5f2729c5c745561f31c9fa505725feab02444b601decf40ba23ef"}
Oct 30 19:37:20.217398 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:20.215689    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-62b91eb5-6579c7c78c-gtv77"
Oct 30 19:37:20.218015 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:20.217969    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-62b91eb5-64bb64b574-5rt9l" event={"ID":"4edc707a-3196-48f1-919a-98e5b12a234e","Type":"ContainerStarted","Data":"5e2002aa9434762a3dc4dfe1007ef3a05b6de1a2941f58d0e4871620a5b94b08"}
Oct 30 19:37:20.218144 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:20.218095    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-62b91eb5-64bb64b574-5rt9l"
Oct 30 19:37:20.222031 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:37:20.222003    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3c0d05d_67f1_4e07_82d0_516ff973b2f2.slice/crio-66185f3cd364f3e6ecf8dd7790138e204874c0a6d4934dbe8f0e76a2d80d6def WatchSource:0}: Error finding container 66185f3cd364f3e6ecf8dd7790138e204874c0a6d4934dbe8f0e76a2d80d6def: Status 404 returned error can't find the container with id 66185f3cd364f3e6ecf8dd7790138e204874c0a6d4934dbe8f0e76a2d80d6def
Oct 30 19:37:20.222245 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:20.222226    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-5016f41a-bff455988-57j6h"]
Oct 30 19:37:20.237253 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:20.236774    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-5016f41a-bd5df848c-mvxr6"]
Oct 30 19:37:20.238016 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:37:20.237990    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7659a68f_a908_46c0_a61c_74f827b84339.slice/crio-670920b46a809adcefc95da9673fa7c5f1d49db3387342f82695ab2c90ec2989 WatchSource:0}: Error finding container 670920b46a809adcefc95da9673fa7c5f1d49db3387342f82695ab2c90ec2989: Status 404 returned error can't find the container with id 670920b46a809adcefc95da9673fa7c5f1d49db3387342f82695ab2c90ec2989
Oct 30 19:37:20.247817 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:20.247795    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3b90b28-28lz8"
Oct 30 19:37:20.267555 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:20.267508    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-62b91eb5-6579c7c78c-gtv77" podStartSLOduration=3.267494094 podStartE2EDuration="3.267494094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:37:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:37:20.261370986 +0000 UTC m=+952.104302924" watchObservedRunningTime="2026-10-30 19:37:20.267494094 +0000 UTC m=+952.110426034"
Oct 30 19:37:20.731306 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:20.696974    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-62b91eb5-64bb64b574-5rt9l" podStartSLOduration=3.69695219 podStartE2EDuration="3.69695219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:37:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:37:20.30835523 +0000 UTC m=+952.151287180" watchObservedRunningTime="2026-10-30 19:37:20.69695219 +0000 UTC m=+952.539884127"
Oct 30 19:37:20.770438 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:20.740189    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e3b90b28-28lz8"]
Oct 30 19:37:20.808313 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:37:20.765804    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c7e341b_211b_4628_a7f6_e96271655434.slice/crio-63534cfdd383aaa2dbde71af8b5cc1c3bb695d7b0a1a3b7db7a3bf765bf83a72 WatchSource:0}: Error finding container 63534cfdd383aaa2dbde71af8b5cc1c3bb695d7b0a1a3b7db7a3bf765bf83a72: Status 404 returned error can't find the container with id 63534cfdd383aaa2dbde71af8b5cc1c3bb695d7b0a1a3b7db7a3bf765bf83a72
Oct 30 19:37:21.229129 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:21.229094    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5016f41a-bd5df848c-mvxr6" event={"ID":"7659a68f-a908-46c0-a61c-74f827b84339","Type":"ContainerStarted","Data":"b09bd77b046c684d36c7ea545c0821bbd163558ed6b3f16b47b6b25a5b04be29"}
Oct 30 19:37:21.229610 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:21.229156    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5016f41a-bd5df848c-mvxr6" event={"ID":"7659a68f-a908-46c0-a61c-74f827b84339","Type":"ContainerStarted","Data":"670920b46a809adcefc95da9673fa7c5f1d49db3387342f82695ab2c90ec2989"}
Oct 30 19:37:21.229610 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:21.229194    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5016f41a-bd5df848c-mvxr6"
Oct 30 19:37:21.231160 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:21.231135    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-5016f41a-bff455988-57j6h" event={"ID":"a3c0d05d-67f1-4e07-82d0-516ff973b2f2","Type":"ContainerStarted","Data":"520e7ae058f45d40c48f1c38ad09df60e8743b57f129a229e4e5ae3c0d1e27cb"}
Oct 30 19:37:21.231303 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:21.231174    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-5016f41a-bff455988-57j6h" event={"ID":"a3c0d05d-67f1-4e07-82d0-516ff973b2f2","Type":"ContainerStarted","Data":"66185f3cd364f3e6ecf8dd7790138e204874c0a6d4934dbe8f0e76a2d80d6def"}
Oct 30 19:37:21.231303 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:21.231201    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-5016f41a-bff455988-57j6h"
Oct 30 19:37:21.232731 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:21.232704    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3b90b28-28lz8" event={"ID":"1c7e341b-211b-4628-a7f6-e96271655434","Type":"ContainerStarted","Data":"63534cfdd383aaa2dbde71af8b5cc1c3bb695d7b0a1a3b7db7a3bf765bf83a72"}
Oct 30 19:37:21.277332 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:21.277272    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5016f41a-bd5df848c-mvxr6" podStartSLOduration=2.277255378 podStartE2EDuration="2.277255378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:37:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:37:21.270905052 +0000 UTC m=+953.113836993" watchObservedRunningTime="2026-10-30 19:37:21.277255378 +0000 UTC m=+953.120187318"
Oct 30 19:37:21.312826 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:21.312767    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-5016f41a-bff455988-57j6h" podStartSLOduration=2.312748397 podStartE2EDuration="2.312748397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:37:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:37:21.311793979 +0000 UTC m=+953.154725919" watchObservedRunningTime="2026-10-30 19:37:21.312748397 +0000 UTC m=+953.155680336"
Oct 30 19:37:21.959398 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:21.959358    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-df776e5c-57c7bd5f5f-rvsvz"]
Oct 30 19:37:21.965807 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:21.965779    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-df776e5c-57c7bd5f5f-rvsvz"
Oct 30 19:37:21.967352 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:21.967328    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-df776e5c-6b7db86b98-xbxfg"]
Oct 30 19:37:21.972118 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:21.972101    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-df776e5c-6b7db86b98-xbxfg"
Oct 30 19:37:21.982340 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:21.982319    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-df776e5c\""
Oct 30 19:37:21.991795 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:21.991769    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-df776e5c-57c7bd5f5f-rvsvz"]
Oct 30 19:37:21.996910 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:21.995940    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-df776e5c-dockercfg-p9wlq\""
Oct 30 19:37:21.999752 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:21.999726    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-df776e5c-6b7db86b98-xbxfg"]
Oct 30 19:37:22.067356 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:22.067324    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/207b8150-f938-4bce-a4d6-473fdf54212d-plugins-config-volume\") pod \"payload-processing-e2e-derive-df776e5c-6b7db86b98-xbxfg\" (UID: \"207b8150-f938-4bce-a4d6-473fdf54212d\") " pod="openshift-ingress/payload-processing-e2e-derive-df776e5c-6b7db86b98-xbxfg"
Oct 30 19:37:22.067489 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:22.067465    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cggx\" (UniqueName: \"kubernetes.io/projected/207b8150-f938-4bce-a4d6-473fdf54212d-kube-api-access-8cggx\") pod \"payload-processing-e2e-derive-df776e5c-6b7db86b98-xbxfg\" (UID: \"207b8150-f938-4bce-a4d6-473fdf54212d\") " pod="openshift-ingress/payload-processing-e2e-derive-df776e5c-6b7db86b98-xbxfg"
Oct 30 19:37:22.067643 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:22.067620    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2c3dee9-2c52-44a2-a64e-aa661cce2452-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-df776e5c-57c7bd5f5f-rvsvz\" (UID: \"f2c3dee9-2c52-44a2-a64e-aa661cce2452\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-df776e5c-57c7bd5f5f-rvsvz"
Oct 30 19:37:22.067706 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:22.067692    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54wd5\" (UniqueName: \"kubernetes.io/projected/f2c3dee9-2c52-44a2-a64e-aa661cce2452-kube-api-access-54wd5\") pod \"payload-pre-processing-e2e-derive-df776e5c-57c7bd5f5f-rvsvz\" (UID: \"f2c3dee9-2c52-44a2-a64e-aa661cce2452\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-df776e5c-57c7bd5f5f-rvsvz"
Oct 30 19:37:22.172290 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:22.172191    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2c3dee9-2c52-44a2-a64e-aa661cce2452-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-df776e5c-57c7bd5f5f-rvsvz\" (UID: \"f2c3dee9-2c52-44a2-a64e-aa661cce2452\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-df776e5c-57c7bd5f5f-rvsvz"
Oct 30 19:37:22.173522 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:22.173495    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54wd5\" (UniqueName: \"kubernetes.io/projected/f2c3dee9-2c52-44a2-a64e-aa661cce2452-kube-api-access-54wd5\") pod \"payload-pre-processing-e2e-derive-df776e5c-57c7bd5f5f-rvsvz\" (UID: \"f2c3dee9-2c52-44a2-a64e-aa661cce2452\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-df776e5c-57c7bd5f5f-rvsvz"
Oct 30 19:37:22.173849 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:22.173831    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/207b8150-f938-4bce-a4d6-473fdf54212d-plugins-config-volume\") pod \"payload-processing-e2e-derive-df776e5c-6b7db86b98-xbxfg\" (UID: \"207b8150-f938-4bce-a4d6-473fdf54212d\") " pod="openshift-ingress/payload-processing-e2e-derive-df776e5c-6b7db86b98-xbxfg"
Oct 30 19:37:22.174139 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:22.174118    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cggx\" (UniqueName: \"kubernetes.io/projected/207b8150-f938-4bce-a4d6-473fdf54212d-kube-api-access-8cggx\") pod \"payload-processing-e2e-derive-df776e5c-6b7db86b98-xbxfg\" (UID: \"207b8150-f938-4bce-a4d6-473fdf54212d\") " pod="openshift-ingress/payload-processing-e2e-derive-df776e5c-6b7db86b98-xbxfg"
Oct 30 19:37:22.183439 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:22.173365    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2c3dee9-2c52-44a2-a64e-aa661cce2452-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-df776e5c-57c7bd5f5f-rvsvz\" (UID: \"f2c3dee9-2c52-44a2-a64e-aa661cce2452\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-df776e5c-57c7bd5f5f-rvsvz"
Oct 30 19:37:22.183610 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:22.175037    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/207b8150-f938-4bce-a4d6-473fdf54212d-plugins-config-volume\") pod \"payload-processing-e2e-derive-df776e5c-6b7db86b98-xbxfg\" (UID: \"207b8150-f938-4bce-a4d6-473fdf54212d\") " pod="openshift-ingress/payload-processing-e2e-derive-df776e5c-6b7db86b98-xbxfg"
Oct 30 19:37:22.225646 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:22.225610    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cggx\" (UniqueName: \"kubernetes.io/projected/207b8150-f938-4bce-a4d6-473fdf54212d-kube-api-access-8cggx\") pod \"payload-processing-e2e-derive-df776e5c-6b7db86b98-xbxfg\" (UID: \"207b8150-f938-4bce-a4d6-473fdf54212d\") " pod="openshift-ingress/payload-processing-e2e-derive-df776e5c-6b7db86b98-xbxfg"
Oct 30 19:37:22.265785 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:22.261006    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-54wd5\" (UniqueName: \"kubernetes.io/projected/f2c3dee9-2c52-44a2-a64e-aa661cce2452-kube-api-access-54wd5\") pod \"payload-pre-processing-e2e-derive-df776e5c-57c7bd5f5f-rvsvz\" (UID: \"f2c3dee9-2c52-44a2-a64e-aa661cce2452\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-df776e5c-57c7bd5f5f-rvsvz"
Oct 30 19:37:22.325907 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:22.317273    5907 generic.go:356] "Generic (PLEG): container finished" podID="1c7e341b-211b-4628-a7f6-e96271655434" containerID="7535bc14346ebcf460fc6daee2c4df11b713378cd598b05e2e45a268f9a1700d" exitCode=0
Oct 30 19:37:22.330374 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:22.329224    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-df776e5c-57c7bd5f5f-rvsvz"
Oct 30 19:37:22.330374 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:22.329652    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-df776e5c-6b7db86b98-xbxfg"
Oct 30 19:37:22.338509 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:22.338472    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3b90b28-28lz8" event={"ID":"1c7e341b-211b-4628-a7f6-e96271655434","Type":"ContainerDied","Data":"7535bc14346ebcf460fc6daee2c4df11b713378cd598b05e2e45a268f9a1700d"}
Oct 30 19:37:22.837416 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:22.837393    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-df776e5c-6b7db86b98-xbxfg"]
Oct 30 19:37:22.839005 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:37:22.838979    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod207b8150_f938_4bce_a4d6_473fdf54212d.slice/crio-fe71d22aa8321d4bbb39de3e0ecad8633640eca9d58b85a55f822c3f63983925 WatchSource:0}: Error finding container fe71d22aa8321d4bbb39de3e0ecad8633640eca9d58b85a55f822c3f63983925: Status 404 returned error can't find the container with id fe71d22aa8321d4bbb39de3e0ecad8633640eca9d58b85a55f822c3f63983925
Oct 30 19:37:22.851507 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:22.851478    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-df776e5c-57c7bd5f5f-rvsvz"]
Oct 30 19:37:22.854239 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:37:22.854213    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2c3dee9_2c52_44a2_a64e_aa661cce2452.slice/crio-0dcbb4362752cca94225e7e16930374daaaad28a531a2d69b0a1bbf347dec41f WatchSource:0}: Error finding container 0dcbb4362752cca94225e7e16930374daaaad28a531a2d69b0a1bbf347dec41f: Status 404 returned error can't find the container with id 0dcbb4362752cca94225e7e16930374daaaad28a531a2d69b0a1bbf347dec41f
Oct 30 19:37:23.336453 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:23.336417    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-df776e5c-6b7db86b98-xbxfg" event={"ID":"207b8150-f938-4bce-a4d6-473fdf54212d","Type":"ContainerStarted","Data":"209f228f2e91ea65e264665577cd1fa7e92de54f272c298e6d1f39b9bfbed352"}
Oct 30 19:37:23.337049 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:23.337026    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-df776e5c-6b7db86b98-xbxfg"
Oct 30 19:37:23.337196 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:23.337175    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-df776e5c-6b7db86b98-xbxfg" event={"ID":"207b8150-f938-4bce-a4d6-473fdf54212d","Type":"ContainerStarted","Data":"fe71d22aa8321d4bbb39de3e0ecad8633640eca9d58b85a55f822c3f63983925"}
Oct 30 19:37:23.338964 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:23.338938    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-df776e5c-57c7bd5f5f-rvsvz" event={"ID":"f2c3dee9-2c52-44a2-a64e-aa661cce2452","Type":"ContainerStarted","Data":"0c14c210a21d68514911ce66e7caa7132f66fbf30f76c2e7c62834fd9e369e9d"}
Oct 30 19:37:23.339088 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:23.338990    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-df776e5c-57c7bd5f5f-rvsvz" event={"ID":"f2c3dee9-2c52-44a2-a64e-aa661cce2452","Type":"ContainerStarted","Data":"0dcbb4362752cca94225e7e16930374daaaad28a531a2d69b0a1bbf347dec41f"}
Oct 30 19:37:23.339224 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:23.339154    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-df776e5c-57c7bd5f5f-rvsvz"
Oct 30 19:37:23.385333 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:23.385270    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-df776e5c-6b7db86b98-xbxfg" podStartSLOduration=2.38525652 podStartE2EDuration="2.38525652s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:37:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:37:23.37842127 +0000 UTC m=+955.221353207" watchObservedRunningTime="2026-10-30 19:37:23.38525652 +0000 UTC m=+955.228188460"
Oct 30 19:37:23.423931 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:23.423852    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-df776e5c-57c7bd5f5f-rvsvz" podStartSLOduration=2.423834369 podStartE2EDuration="2.423834369s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:37:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:37:23.419988227 +0000 UTC m=+955.262920166" watchObservedRunningTime="2026-10-30 19:37:23.423834369 +0000 UTC m=+955.266766310"
Oct 30 19:37:23.518454 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:23.518428    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3b90b28-28lz8"
Oct 30 19:37:23.663887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:23.663760    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1c7e341b-211b-4628-a7f6-e96271655434-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1c7e341b-211b-4628-a7f6-e96271655434-maas-api-openshift-ca\") pod \"1c7e341b-211b-4628-a7f6-e96271655434\" (UID: \"1c7e341b-211b-4628-a7f6-e96271655434\") "
Oct 30 19:37:23.663887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:23.663839    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1c7e341b-211b-4628-a7f6-e96271655434-kube-api-access-lsb7w\" (UniqueName: \"kubernetes.io/projected/1c7e341b-211b-4628-a7f6-e96271655434-kube-api-access-lsb7w\") pod \"1c7e341b-211b-4628-a7f6-e96271655434\" (UID: \"1c7e341b-211b-4628-a7f6-e96271655434\") "
Oct 30 19:37:23.705023 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:23.704984    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1c7e341b-211b-4628-a7f6-e96271655434-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1c7e341b-211b-4628-a7f6-e96271655434-maas-api-xks-ca\") pod \"1c7e341b-211b-4628-a7f6-e96271655434\" (UID: \"1c7e341b-211b-4628-a7f6-e96271655434\") "
Oct 30 19:37:23.709758 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:23.706163    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c7e341b-211b-4628-a7f6-e96271655434-kube-api-access-lsb7w" pod "1c7e341b-211b-4628-a7f6-e96271655434" (UID: "1c7e341b-211b-4628-a7f6-e96271655434"). InnerVolumeSpecName "kube-api-access-lsb7w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:37:23.709758 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:23.706486    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c7e341b-211b-4628-a7f6-e96271655434-maas-api-openshift-ca" pod "1c7e341b-211b-4628-a7f6-e96271655434" (UID: "1c7e341b-211b-4628-a7f6-e96271655434"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:37:23.709758 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:23.707984    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1c7e341b-211b-4628-a7f6-e96271655434-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:37:23.709758 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:23.708007    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lsb7w\" (UniqueName: \"kubernetes.io/projected/1c7e341b-211b-4628-a7f6-e96271655434-kube-api-access-lsb7w\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:37:23.712588 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:23.712355    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c7e341b-211b-4628-a7f6-e96271655434-maas-api-xks-ca" pod "1c7e341b-211b-4628-a7f6-e96271655434" (UID: "1c7e341b-211b-4628-a7f6-e96271655434"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:37:23.809409 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:23.809378    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1c7e341b-211b-4628-a7f6-e96271655434-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:37:24.350347 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:24.350286    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3b90b28-28lz8"
Oct 30 19:37:24.350347 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:24.350317    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3b90b28-28lz8" event={"ID":"1c7e341b-211b-4628-a7f6-e96271655434","Type":"ContainerDied","Data":"63534cfdd383aaa2dbde71af8b5cc1c3bb695d7b0a1a3b7db7a3bf765bf83a72"}
Oct 30 19:37:24.350827 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:24.350373    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="63534cfdd383aaa2dbde71af8b5cc1c3bb695d7b0a1a3b7db7a3bf765bf83a72"
Oct 30 19:37:31.005757 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.005717    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"]
Oct 30 19:37:31.006904 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.006835    5907 cpu_manager.go:414] "containerMap: removing container" podUID="1c7e341b-211b-4628-a7f6-e96271655434" containerName="revoke-keys"
Oct 30 19:37:31.006904 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.006877    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1c7e341b-211b-4628-a7f6-e96271655434" containerName="revoke-keys"
Oct 30 19:37:31.007124 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.007108    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="1c7e341b-211b-4628-a7f6-e96271655434" containerName="revoke-keys"
Oct 30 19:37:31.013817 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.013602    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"
Oct 30 19:37:31.038658 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.036110    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"]
Oct 30 19:37:31.081806 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.081764    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2lb5\" (UniqueName: \"kubernetes.io/projected/ddf16588-cce5-46af-8ff3-e927fb5d76b5-kube-api-access-g2lb5\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"
Oct 30 19:37:31.081998 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.081926    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddf16588-cce5-46af-8ff3-e927fb5d76b5-workload-socket\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"
Oct 30 19:37:31.081998 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.081985    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddf16588-cce5-46af-8ff3-e927fb5d76b5-istio-data\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"
Oct 30 19:37:31.082179 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.082160    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddf16588-cce5-46af-8ff3-e927fb5d76b5-credential-socket\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"
Oct 30 19:37:31.082306 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.082291    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddf16588-cce5-46af-8ff3-e927fb5d76b5-workload-certs\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"
Oct 30 19:37:31.082370 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.082348    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddf16588-cce5-46af-8ff3-e927fb5d76b5-istiod-ca-cert\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"
Oct 30 19:37:31.082457 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.082430    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddf16588-cce5-46af-8ff3-e927fb5d76b5-istio-token\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"
Oct 30 19:37:31.082528 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.082511    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddf16588-cce5-46af-8ff3-e927fb5d76b5-istio-podinfo\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"
Oct 30 19:37:31.082583 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.082569    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddf16588-cce5-46af-8ff3-e927fb5d76b5-istio-envoy\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"
Oct 30 19:37:31.188791 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.188743    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddf16588-cce5-46af-8ff3-e927fb5d76b5-workload-certs\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"
Oct 30 19:37:31.189144 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.189119    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddf16588-cce5-46af-8ff3-e927fb5d76b5-istiod-ca-cert\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"
Oct 30 19:37:31.189305 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.189290    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddf16588-cce5-46af-8ff3-e927fb5d76b5-istio-token\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"
Oct 30 19:37:31.189446 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.189434    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddf16588-cce5-46af-8ff3-e927fb5d76b5-istio-podinfo\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"
Oct 30 19:37:31.189608 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.189588    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddf16588-cce5-46af-8ff3-e927fb5d76b5-istio-envoy\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"
Oct 30 19:37:31.189835 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.189814    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2lb5\" (UniqueName: \"kubernetes.io/projected/ddf16588-cce5-46af-8ff3-e927fb5d76b5-kube-api-access-g2lb5\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"
Oct 30 19:37:31.193106 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.190071    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddf16588-cce5-46af-8ff3-e927fb5d76b5-workload-socket\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"
Oct 30 19:37:31.193337 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.193311    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddf16588-cce5-46af-8ff3-e927fb5d76b5-istio-data\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"
Oct 30 19:37:31.193602 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.193583    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddf16588-cce5-46af-8ff3-e927fb5d76b5-credential-socket\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"
Oct 30 19:37:31.194362 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.194307    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddf16588-cce5-46af-8ff3-e927fb5d76b5-credential-socket\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"
Oct 30 19:37:31.197958 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.197934    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddf16588-cce5-46af-8ff3-e927fb5d76b5-istiod-ca-cert\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"
Oct 30 19:37:31.205135 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.205079    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddf16588-cce5-46af-8ff3-e927fb5d76b5-workload-certs\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"
Oct 30 19:37:31.211688 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.206352    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddf16588-cce5-46af-8ff3-e927fb5d76b5-workload-socket\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"
Oct 30 19:37:31.211929 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.206599    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddf16588-cce5-46af-8ff3-e927fb5d76b5-istio-data\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"
Oct 30 19:37:31.213548 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.213519    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddf16588-cce5-46af-8ff3-e927fb5d76b5-istio-podinfo\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"
Oct 30 19:37:31.215173 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.215147    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddf16588-cce5-46af-8ff3-e927fb5d76b5-istio-envoy\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"
Oct 30 19:37:31.221703 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.221680    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddf16588-cce5-46af-8ff3-e927fb5d76b5-istio-token\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"
Oct 30 19:37:31.231720 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.231687    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2lb5\" (UniqueName: \"kubernetes.io/projected/ddf16588-cce5-46af-8ff3-e927fb5d76b5-kube-api-access-g2lb5\") pod \"e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") " pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"
Oct 30 19:37:31.236910 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.236015    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-62b91eb5-64bb64b574-5rt9l"
Oct 30 19:37:31.236910 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.236459    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-62b91eb5-6579c7c78c-gtv77"
Oct 30 19:37:31.331483 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.331394    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"
Oct 30 19:37:31.886310 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:37:31.886280    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddf16588_cce5_46af_8ff3_e927fb5d76b5.slice/crio-0d04eddde29796b788d4af2955db8df07d933e98d039e6340c309ef877a3861a WatchSource:0}: Error finding container 0d04eddde29796b788d4af2955db8df07d933e98d039e6340c309ef877a3861a: Status 404 returned error can't find the container with id 0d04eddde29796b788d4af2955db8df07d933e98d039e6340c309ef877a3861a
Oct 30 19:37:31.889168 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.889139    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"]
Oct 30 19:37:31.889556 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.889524    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:37:31.889610 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.889586    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:37:31.889646 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:31.889615    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:37:32.181625 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.181505    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"]
Oct 30 19:37:32.188895 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.188850    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"
Oct 30 19:37:32.204204 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.204079    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"]
Oct 30 19:37:32.208151 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.208125    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-46bd1516-openshift-default-dockercfg-hpq5f\""
Oct 30 19:37:32.234284 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.234239    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbdba47e-927d-454e-b860-ed6ea071304b-istio-podinfo\") pod \"e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") " pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"
Oct 30 19:37:32.234473 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.234300    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbdba47e-927d-454e-b860-ed6ea071304b-credential-socket\") pod \"e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") " pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"
Oct 30 19:37:32.234473 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.234412    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbdba47e-927d-454e-b860-ed6ea071304b-workload-certs\") pod \"e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") " pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"
Oct 30 19:37:32.234604 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.234476    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbdba47e-927d-454e-b860-ed6ea071304b-istio-data\") pod \"e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") " pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"
Oct 30 19:37:32.234604 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.234546    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbdba47e-927d-454e-b860-ed6ea071304b-istio-token\") pod \"e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") " pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"
Oct 30 19:37:32.234727 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.234624    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpk8v\" (UniqueName: \"kubernetes.io/projected/fbdba47e-927d-454e-b860-ed6ea071304b-kube-api-access-kpk8v\") pod \"e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") " pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"
Oct 30 19:37:32.234727 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.234669    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbdba47e-927d-454e-b860-ed6ea071304b-istiod-ca-cert\") pod \"e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") " pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"
Oct 30 19:37:32.234839 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.234794    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbdba47e-927d-454e-b860-ed6ea071304b-workload-socket\") pod \"e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") " pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"
Oct 30 19:37:32.235038 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.235017    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbdba47e-927d-454e-b860-ed6ea071304b-istio-envoy\") pod \"e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") " pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"
Oct 30 19:37:32.322308 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.322278    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5016f41a-bd5df848c-mvxr6"
Oct 30 19:37:32.361138 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.361093    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbdba47e-927d-454e-b860-ed6ea071304b-istio-token\") pod \"e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") " pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"
Oct 30 19:37:32.361460 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.361422    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpk8v\" (UniqueName: \"kubernetes.io/projected/fbdba47e-927d-454e-b860-ed6ea071304b-kube-api-access-kpk8v\") pod \"e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") " pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"
Oct 30 19:37:32.361665 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.361632    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbdba47e-927d-454e-b860-ed6ea071304b-istiod-ca-cert\") pod \"e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") " pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"
Oct 30 19:37:32.361920 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.361900    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbdba47e-927d-454e-b860-ed6ea071304b-workload-socket\") pod \"e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") " pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"
Oct 30 19:37:32.362209 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.362192    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbdba47e-927d-454e-b860-ed6ea071304b-istio-envoy\") pod \"e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") " pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"
Oct 30 19:37:32.362523 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.362507    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbdba47e-927d-454e-b860-ed6ea071304b-istio-podinfo\") pod \"e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") " pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"
Oct 30 19:37:32.362689 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.362674    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbdba47e-927d-454e-b860-ed6ea071304b-credential-socket\") pod \"e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") " pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"
Oct 30 19:37:32.362910 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.362883    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbdba47e-927d-454e-b860-ed6ea071304b-istiod-ca-cert\") pod \"e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") " pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"
Oct 30 19:37:32.363011 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.362886    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbdba47e-927d-454e-b860-ed6ea071304b-workload-certs\") pod \"e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") " pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"
Oct 30 19:37:32.363079 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.363007    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbdba47e-927d-454e-b860-ed6ea071304b-istio-data\") pod \"e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") " pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"
Oct 30 19:37:32.363892 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.363478    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbdba47e-927d-454e-b860-ed6ea071304b-istio-data\") pod \"e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") " pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"
Oct 30 19:37:32.363892 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.363738    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbdba47e-927d-454e-b860-ed6ea071304b-credential-socket\") pod \"e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") " pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"
Oct 30 19:37:32.378801 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.363989    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbdba47e-927d-454e-b860-ed6ea071304b-workload-socket\") pod \"e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") " pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"
Oct 30 19:37:32.379745 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.379634    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbdba47e-927d-454e-b860-ed6ea071304b-workload-certs\") pod \"e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") " pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"
Oct 30 19:37:32.406438 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.404735    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbdba47e-927d-454e-b860-ed6ea071304b-istio-envoy\") pod \"e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") " pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"
Oct 30 19:37:32.451683 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.442586    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpk8v\" (UniqueName: \"kubernetes.io/projected/fbdba47e-927d-454e-b860-ed6ea071304b-kube-api-access-kpk8v\") pod \"e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") " pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"
Oct 30 19:37:32.451683 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.451454    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbdba47e-927d-454e-b860-ed6ea071304b-istio-podinfo\") pod \"e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") " pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"
Oct 30 19:37:32.452137 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.452118    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-5016f41a-bff455988-57j6h"
Oct 30 19:37:32.718957 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.716602    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbdba47e-927d-454e-b860-ed6ea071304b-istio-token\") pod \"e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") " pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"
Oct 30 19:37:32.745499 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.744917    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s" event={"ID":"ddf16588-cce5-46af-8ff3-e927fb5d76b5","Type":"ContainerStarted","Data":"581fa8b5cf363fd38edafe510db0b783bead95d822bd1496ca2c678f13e29ec9"}
Oct 30 19:37:32.745499 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.744985    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s" event={"ID":"ddf16588-cce5-46af-8ff3-e927fb5d76b5","Type":"ContainerStarted","Data":"0d04eddde29796b788d4af2955db8df07d933e98d039e6340c309ef877a3861a"}
Oct 30 19:37:32.838655 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.838588    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s" podStartSLOduration=2.83856729 podStartE2EDuration="2.83856729s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:37:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:37:32.831270029 +0000 UTC m=+964.674202009" watchObservedRunningTime="2026-10-30 19:37:32.83856729 +0000 UTC m=+964.681499231"
Oct 30 19:37:32.840648 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:32.840598    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"
Oct 30 19:37:33.337169 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:33.332155    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"
Oct 30 19:37:33.339221 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:33.338378    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:37:33.339221 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:33.338478    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:37:33.339221 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:33.338520    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:37:33.342139 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:33.342009    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused" start-of-body=
Oct 30 19:37:33.342139 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:33.342062    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s" podUID="ddf16588-cce5-46af-8ff3-e927fb5d76b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused"
Oct 30 19:37:33.369711 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:33.368038    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"]
Oct 30 19:37:33.770999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:33.767518    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw" event={"ID":"fbdba47e-927d-454e-b860-ed6ea071304b","Type":"ContainerStarted","Data":"8e91f00589b8503459bc781a01f5c186ac5e69f9dd3a7cbc8eb87fdd9430ae67"}
Oct 30 19:37:33.770999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:33.767589    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw" event={"ID":"fbdba47e-927d-454e-b860-ed6ea071304b","Type":"ContainerStarted","Data":"0ab66b26b47f44fac1bbe34528c99b880fbe33438b3391d97228d704986956d6"}
Oct 30 19:37:33.843812 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:33.843779    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"
Oct 30 19:37:34.336029 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:34.335966    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused" start-of-body=
Oct 30 19:37:34.336259 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:34.336049    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s" podUID="ddf16588-cce5-46af-8ff3-e927fb5d76b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused"
Oct 30 19:37:34.353925 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:34.353769    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-df776e5c-6b7db86b98-xbxfg"
Oct 30 19:37:34.353925 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:34.353881    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-df776e5c-57c7bd5f5f-rvsvz"
Oct 30 19:37:34.426890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:34.410632    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw" podStartSLOduration=2.41060934 podStartE2EDuration="2.41060934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:37:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:37:33.838103142 +0000 UTC m=+965.681035082" watchObservedRunningTime="2026-10-30 19:37:34.41060934 +0000 UTC m=+966.253541282"
Oct 30 19:37:34.851035 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:34.851004    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"
Oct 30 19:37:35.334321 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:35.334279    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused" start-of-body=
Oct 30 19:37:35.334629 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:35.334598    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s" podUID="ddf16588-cce5-46af-8ff3-e927fb5d76b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused"
Oct 30 19:37:35.809723 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:35.809691    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"
Oct 30 19:37:35.835950 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:35.835372    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"
Oct 30 19:37:36.334447 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:36.333981    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused" start-of-body=
Oct 30 19:37:36.334447 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:36.334048    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s" podUID="ddf16588-cce5-46af-8ff3-e927fb5d76b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused"
Oct 30 19:37:37.334888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:37.332314    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused" start-of-body=
Oct 30 19:37:37.334888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:37.332377    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s" podUID="ddf16588-cce5-46af-8ff3-e927fb5d76b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused"
Oct 30 19:37:37.435917 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:37.431088    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:37:38.332362 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:38.332324    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused" start-of-body=
Oct 30 19:37:38.332644 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:38.332611    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s" podUID="ddf16588-cce5-46af-8ff3-e927fb5d76b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused"
Oct 30 19:37:38.660964 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:38.660911    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-46bd1516-b8686b4d7-w8mcm"]
Oct 30 19:37:38.667543 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:38.667525    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-46bd1516-b8686b4d7-w8mcm"
Oct 30 19:37:38.691104 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:38.691081    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-46bd1516-b8686b4d7-w8mcm"]
Oct 30 19:37:38.695720 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:38.695699    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-46bd1516-dockercfg-rqc2r\""
Oct 30 19:37:38.695887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:38.695699    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-46bd1516\""
Oct 30 19:37:38.706685 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:38.706659    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-46bd1516-6d7b686fb-rfggk"]
Oct 30 19:37:38.750564 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:38.750528    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-46bd1516-6d7b686fb-rfggk"
Oct 30 19:37:38.764242 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:38.764206    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bea8608e-101a-4aac-8164-9d757d8281c9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-46bd1516-b8686b4d7-w8mcm\" (UID: \"bea8608e-101a-4aac-8164-9d757d8281c9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-46bd1516-b8686b4d7-w8mcm"
Oct 30 19:37:38.764530 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:38.764496    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szzv7\" (UniqueName: \"kubernetes.io/projected/bea8608e-101a-4aac-8164-9d757d8281c9-kube-api-access-szzv7\") pod \"payload-pre-processing-e2e-ait-46bd1516-b8686b4d7-w8mcm\" (UID: \"bea8608e-101a-4aac-8164-9d757d8281c9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-46bd1516-b8686b4d7-w8mcm"
Oct 30 19:37:38.779983 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:38.779946    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-46bd1516-6d7b686fb-rfggk"]
Oct 30 19:37:38.890097 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:38.890061    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kd286\" (UniqueName: \"kubernetes.io/projected/64d0a763-2322-4d30-9745-c650d15e1d05-kube-api-access-kd286\") pod \"payload-processing-e2e-ait-46bd1516-6d7b686fb-rfggk\" (UID: \"64d0a763-2322-4d30-9745-c650d15e1d05\") " pod="openshift-ingress/payload-processing-e2e-ait-46bd1516-6d7b686fb-rfggk"
Oct 30 19:37:38.890608 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:38.890523    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bea8608e-101a-4aac-8164-9d757d8281c9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-46bd1516-b8686b4d7-w8mcm\" (UID: \"bea8608e-101a-4aac-8164-9d757d8281c9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-46bd1516-b8686b4d7-w8mcm"
Oct 30 19:37:38.890932 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:38.890916    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szzv7\" (UniqueName: \"kubernetes.io/projected/bea8608e-101a-4aac-8164-9d757d8281c9-kube-api-access-szzv7\") pod \"payload-pre-processing-e2e-ait-46bd1516-b8686b4d7-w8mcm\" (UID: \"bea8608e-101a-4aac-8164-9d757d8281c9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-46bd1516-b8686b4d7-w8mcm"
Oct 30 19:37:38.965056 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:38.962027    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64d0a763-2322-4d30-9745-c650d15e1d05-plugins-config-volume\") pod \"payload-processing-e2e-ait-46bd1516-6d7b686fb-rfggk\" (UID: \"64d0a763-2322-4d30-9745-c650d15e1d05\") " pod="openshift-ingress/payload-processing-e2e-ait-46bd1516-6d7b686fb-rfggk"
Oct 30 19:37:38.965056 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:38.937845    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bea8608e-101a-4aac-8164-9d757d8281c9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-46bd1516-b8686b4d7-w8mcm\" (UID: \"bea8608e-101a-4aac-8164-9d757d8281c9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-46bd1516-b8686b4d7-w8mcm"
Oct 30 19:37:39.022837 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:39.022795    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-szzv7\" (UniqueName: \"kubernetes.io/projected/bea8608e-101a-4aac-8164-9d757d8281c9-kube-api-access-szzv7\") pod \"payload-pre-processing-e2e-ait-46bd1516-b8686b4d7-w8mcm\" (UID: \"bea8608e-101a-4aac-8164-9d757d8281c9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-46bd1516-b8686b4d7-w8mcm"
Oct 30 19:37:39.091885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:39.067207    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64d0a763-2322-4d30-9745-c650d15e1d05-plugins-config-volume\") pod \"payload-processing-e2e-ait-46bd1516-6d7b686fb-rfggk\" (UID: \"64d0a763-2322-4d30-9745-c650d15e1d05\") " pod="openshift-ingress/payload-processing-e2e-ait-46bd1516-6d7b686fb-rfggk"
Oct 30 19:37:39.091885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:39.067514    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kd286\" (UniqueName: \"kubernetes.io/projected/64d0a763-2322-4d30-9745-c650d15e1d05-kube-api-access-kd286\") pod \"payload-processing-e2e-ait-46bd1516-6d7b686fb-rfggk\" (UID: \"64d0a763-2322-4d30-9745-c650d15e1d05\") " pod="openshift-ingress/payload-processing-e2e-ait-46bd1516-6d7b686fb-rfggk"
Oct 30 19:37:39.093454 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:39.093386    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64d0a763-2322-4d30-9745-c650d15e1d05-plugins-config-volume\") pod \"payload-processing-e2e-ait-46bd1516-6d7b686fb-rfggk\" (UID: \"64d0a763-2322-4d30-9745-c650d15e1d05\") " pod="openshift-ingress/payload-processing-e2e-ait-46bd1516-6d7b686fb-rfggk"
Oct 30 19:37:39.139897 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:39.124303    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kd286\" (UniqueName: \"kubernetes.io/projected/64d0a763-2322-4d30-9745-c650d15e1d05-kube-api-access-kd286\") pod \"payload-processing-e2e-ait-46bd1516-6d7b686fb-rfggk\" (UID: \"64d0a763-2322-4d30-9745-c650d15e1d05\") " pod="openshift-ingress/payload-processing-e2e-ait-46bd1516-6d7b686fb-rfggk"
Oct 30 19:37:39.337940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:39.308179    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-46bd1516-b8686b4d7-w8mcm"
Oct 30 19:37:39.339900 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:39.339843    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused" start-of-body=
Oct 30 19:37:39.351882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:39.340165    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s" podUID="ddf16588-cce5-46af-8ff3-e927fb5d76b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused"
Oct 30 19:37:39.459963 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:39.459932    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-46bd1516-6d7b686fb-rfggk"
Oct 30 19:37:39.864963 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:39.864622    5907 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:40.141930 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:40.139369    5907 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:40.368020 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:40.367964    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:40.505940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:40.505010    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused" start-of-body=
Oct 30 19:37:40.505940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:40.505077    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s" podUID="ddf16588-cce5-46af-8ff3-e927fb5d76b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused"
Oct 30 19:37:40.573655 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:40.573597    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": context deadline exceeded"
Oct 30 19:37:41.043711 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:41.007738    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:41.593929 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:41.573525    5907 patch_prober.go:29] interesting pod/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:37:41.593929 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:41.573588    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv" podUID="1a16775f-a08c-433c-8707-9eed8d73e7f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:41.658496 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:41.645911    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused" start-of-body=
Oct 30 19:37:41.682240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:41.682187    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s" podUID="ddf16588-cce5-46af-8ff3-e927fb5d76b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused"
Oct 30 19:37:41.725995 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:41.693790    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 19:37:41.848601 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:41.779981    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:37:41.848601 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:41.727273    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": context deadline exceeded"
Oct 30 19:37:41.848601 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:41.789669    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:37:41.849407 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:41.849079    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:42.045935 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:42.031568    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:42.045935 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:42.032284    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:37:42.361083 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:42.361021    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused" start-of-body=
Oct 30 19:37:42.361426 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:42.361371    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s" podUID="ddf16588-cce5-46af-8ff3-e927fb5d76b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused"
Oct 30 19:37:42.801898 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:42.790025    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:42.801898 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:42.790525    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:42.968474 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:42.968423    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:43.078793 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:43.078688    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:43.338218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:43.338126    5907 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:44.182585 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:44.182530    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:44.268899 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:44.258381    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": context deadline exceeded"
Oct 30 19:37:44.337966 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:44.271518    5907 patch_prober.go:29] interesting pod/metrics-server-6b87cd958c-prt2j container/metrics-server namespace/openshift-monitoring: Readiness probe status=failure output="Get \"https://10.134.0.14:10250/livez\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:37:44.337966 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:44.271585    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-monitoring/metrics-server-6b87cd958c-prt2j" podUID="2ca12196-1929-4ece-a076-d36527b95ff1" containerName="metrics-server" probeResult="failure" output="Get \"https://10.134.0.14:10250/livez\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:44.337966 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:44.271659    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": context deadline exceeded"
Oct 30 19:37:44.337966 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:44.315034    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 19:37:44.664939 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:44.654458    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:37:44.714942 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:37:44.682374    5907 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="2.394s"
Oct 30 19:37:44.803161 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:44.803119    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s" podUID="ddf16588-cce5-46af-8ff3-e927fb5d76b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:45.203172 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:45.203065    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:45.275908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:45.260811    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:45.275908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:45.275002    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:45.346400 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:45.346359    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 19:37:45.442704 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:45.442664    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-46bd1516-b8686b4d7-w8mcm"]
Oct 30 19:37:45.444816 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:45.443064    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:37:45.483937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:45.483825    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-46bd1516-6d7b686fb-rfggk"]
Oct 30 19:37:45.498358 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:45.498325    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-46bd1516-b8686b4d7-w8mcm" event={"ID":"bea8608e-101a-4aac-8164-9d757d8281c9","Type":"ContainerStarted","Data":"5bbf5d4b03e16af8e7987efe644ec5e1ef4545795f915feff4e7b76ace0d77d8"}
Oct 30 19:37:45.941937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:45.929011    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:37:45.941937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:45.929085    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s" podUID="ddf16588-cce5-46af-8ff3-e927fb5d76b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:46.422460 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:46.355356    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:46.447908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:46.359155    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 19:37:46.485953 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:46.472758    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"
Oct 30 19:37:46.485953 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:46.475161    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:46.485953 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:46.475540    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:46.621909 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:46.621789    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-46bd1516-6d7b686fb-rfggk" event={"ID":"64d0a763-2322-4d30-9745-c650d15e1d05","Type":"ContainerStarted","Data":"343a862edead27e3a395297479e3a42c040e8fe240d52538ffb8e3ec69bbae13"}
Oct 30 19:37:46.657327 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:46.657287    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-46bd1516-b8686b4d7-w8mcm" event={"ID":"bea8608e-101a-4aac-8164-9d757d8281c9","Type":"ContainerStarted","Data":"859a231589d446e3d57d72672140943aeb9e22e2211b2f9b180a3ddaa8b2d629"}
Oct 30 19:37:46.663098 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:46.663069    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-46bd1516-b8686b4d7-w8mcm"
Oct 30 19:37:46.743978 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:46.738219    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-46bd1516-b8686b4d7-w8mcm" podStartSLOduration=8.738194305 podStartE2EDuration="8.738194305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:37:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:37:46.703267183 +0000 UTC m=+978.546199123" watchObservedRunningTime="2026-10-30 19:37:46.738194305 +0000 UTC m=+978.581126245"
Oct 30 19:37:46.991996 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:46.976023    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:37:46.991996 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:46.976109    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s" podUID="ddf16588-cce5-46af-8ff3-e927fb5d76b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:47.471054 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:47.471001    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:47.542907 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:47.498875    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 19:37:47.542907 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:47.500536    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": context deadline exceeded"
Oct 30 19:37:47.542907 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:47.500903    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": context deadline exceeded"
Oct 30 19:37:47.810313 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:47.810149    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-46bd1516-6d7b686fb-rfggk" event={"ID":"64d0a763-2322-4d30-9745-c650d15e1d05","Type":"ContainerStarted","Data":"1992630853dd50061c0d8c753b411dcdcec0c7cd32a877c7ff052e9fad5c2293"}
Oct 30 19:37:47.810761 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:47.810707    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-46bd1516-6d7b686fb-rfggk"
Oct 30 19:37:47.874135 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:47.874072    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-46bd1516-6d7b686fb-rfggk" podStartSLOduration=9.874050057 podStartE2EDuration="9.874050057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:37:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:37:47.872287389 +0000 UTC m=+979.715219329" watchObservedRunningTime="2026-10-30 19:37:47.874050057 +0000 UTC m=+979.716981993"
Oct 30 19:37:47.890486 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:37:47.890427    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:37:48.006038 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:48.005996    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:37:48.008096 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:48.008060    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s" podUID="ddf16588-cce5-46af-8ff3-e927fb5d76b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:48.443383 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:48.443004    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:37:48.443383 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:48.443072    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc" podUID="f67f5efb-73b5-49fa-b044-36471405ce25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:48.831948 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:48.807292    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:48.936883 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:48.832715    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 19:37:49.314977 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:48.830890    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": context deadline exceeded"
Oct 30 19:37:49.323025 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:48.902226    5907 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:49.333296 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:48.816550    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-c2sv6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:37:49.349888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:49.333521    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6" podUID="ffa8893a-530b-4232-8e6e-dbe34b30ccda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:49.361889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:49.050586    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:37:49.361889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:49.353088    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s" podUID="ddf16588-cce5-46af-8ff3-e927fb5d76b5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:37:49.363144 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:49.078996    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-s826m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:37:49.363144 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:49.362318    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m" podUID="86efaf93-7f28-4e6a-9add-c8bdd87c5261" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:49.363144 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:49.079077    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:49.560567 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:49.524477    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-pdg78 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:37:49.560567 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:49.524556    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78" podUID="bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:49.766895 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:37:49.660479    5907 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.307s"
Oct 30 19:37:49.857937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:49.838846    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": context deadline exceeded"
Oct 30 19:37:49.877478 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:49.845438    5907 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/health\": context deadline exceeded"
Oct 30 19:37:50.331888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:50.328710    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:50.331888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:50.329124    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 19:37:50.364012 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:50.363961    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 19:37:50.392789 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:50.378961    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:37:50.392789 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:50.379026    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s" podUID="ddf16588-cce5-46af-8ff3-e927fb5d76b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:50.397884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:50.397820    5907 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:50.959300 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:50.959259    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-9tdql container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:37:50.993394 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:50.982113    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql" podUID="8db50859-98fc-49aa-806b-4fbed56b1d00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:51.201914 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:50.972910    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:51.412951 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:51.341384    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": context deadline exceeded"
Oct 30 19:37:51.412951 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:51.341643    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 19:37:51.412951 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:51.396439    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 19:37:51.437915 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:51.434008    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:37:51.437915 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:51.434092    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s" podUID="ddf16588-cce5-46af-8ff3-e927fb5d76b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:52.277025 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:52.276977    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": context deadline exceeded"
Oct 30 19:37:52.505302 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:52.505254    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 19:37:52.580952 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:52.505593    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:52.580952 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:52.505632    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:52.580952 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:52.506814    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:37:52.580952 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:52.575708    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s" podUID="ddf16588-cce5-46af-8ff3-e927fb5d76b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:37:52.759577 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:52.759546    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:37:52.760054 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:52.760037    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"
Oct 30 19:37:52.760235 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:52.760221    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:37:52.760372 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:52.760360    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:37:52.857317 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:52.857232    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:37:52.991017 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:52.990913    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"
Oct 30 19:37:54.046108 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:54.046073    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6b64d8bb-rpw59"]
Oct 30 19:37:54.058639 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:54.058610    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6b64d8bb-rpw59"]
Oct 30 19:37:54.293445 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:54.293410    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8c883612-2ee9-49ed-a69d-d3dce0011061" path="/var/lib/kubelet/pods/8c883612-2ee9-49ed-a69d-d3dce0011061/volumes"
Oct 30 19:37:59.738836 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:59.738798    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-46bd1516-6d7b686fb-rfggk"
Oct 30 19:37:59.739639 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:37:59.739618    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-46bd1516-b8686b4d7-w8mcm"
Oct 30 19:38:02.051485 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:02.051401    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d8ac22a1-xm87l"]
Oct 30 19:38:02.064389 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:02.064334    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d8ac22a1-xm87l"]
Oct 30 19:38:02.291697 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:02.291666    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="57f4366c-c0d7-4b52-88aa-f57069f2f596" path="/var/lib/kubelet/pods/57f4366c-c0d7-4b52-88aa-f57069f2f596/volumes"
Oct 30 19:38:07.394832 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:07.394799    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-62b91eb5-c52kj"]
Oct 30 19:38:07.401701 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:07.401678    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-62b91eb5-c52kj"
Oct 30 19:38:07.421448 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:07.421413    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-62b91eb5-c52kj"]
Oct 30 19:38:07.424572 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:07.424523    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:38:07.424712 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:07.424596    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:38:07.521424 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:07.521388    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dcbde9d9-ab14-457e-bf8f-fdef4de083e0-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-62b91eb5-c52kj\" (UID: \"dcbde9d9-ab14-457e-bf8f-fdef4de083e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-62b91eb5-c52kj"
Oct 30 19:38:07.521599 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:07.521584    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dcbde9d9-ab14-457e-bf8f-fdef4de083e0-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-62b91eb5-c52kj\" (UID: \"dcbde9d9-ab14-457e-bf8f-fdef4de083e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-62b91eb5-c52kj"
Oct 30 19:38:07.521752 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:07.521727    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fs6vx\" (UniqueName: \"kubernetes.io/projected/dcbde9d9-ab14-457e-bf8f-fdef4de083e0-kube-api-access-fs6vx\") pod \"maas-api-revoke-keys-e2e-ait-62b91eb5-c52kj\" (UID: \"dcbde9d9-ab14-457e-bf8f-fdef4de083e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-62b91eb5-c52kj"
Oct 30 19:38:07.623278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:07.623248    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dcbde9d9-ab14-457e-bf8f-fdef4de083e0-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-62b91eb5-c52kj\" (UID: \"dcbde9d9-ab14-457e-bf8f-fdef4de083e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-62b91eb5-c52kj"
Oct 30 19:38:07.623457 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:07.623391    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fs6vx\" (UniqueName: \"kubernetes.io/projected/dcbde9d9-ab14-457e-bf8f-fdef4de083e0-kube-api-access-fs6vx\") pod \"maas-api-revoke-keys-e2e-ait-62b91eb5-c52kj\" (UID: \"dcbde9d9-ab14-457e-bf8f-fdef4de083e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-62b91eb5-c52kj"
Oct 30 19:38:07.623665 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:07.623638    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dcbde9d9-ab14-457e-bf8f-fdef4de083e0-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-62b91eb5-c52kj\" (UID: \"dcbde9d9-ab14-457e-bf8f-fdef4de083e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-62b91eb5-c52kj"
Oct 30 19:38:07.624359 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:07.624334    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dcbde9d9-ab14-457e-bf8f-fdef4de083e0-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-62b91eb5-c52kj\" (UID: \"dcbde9d9-ab14-457e-bf8f-fdef4de083e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-62b91eb5-c52kj"
Oct 30 19:38:07.625804 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:07.625783    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dcbde9d9-ab14-457e-bf8f-fdef4de083e0-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-62b91eb5-c52kj\" (UID: \"dcbde9d9-ab14-457e-bf8f-fdef4de083e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-62b91eb5-c52kj"
Oct 30 19:38:07.645116 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:07.645062    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fs6vx\" (UniqueName: \"kubernetes.io/projected/dcbde9d9-ab14-457e-bf8f-fdef4de083e0-kube-api-access-fs6vx\") pod \"maas-api-revoke-keys-e2e-ait-62b91eb5-c52kj\" (UID: \"dcbde9d9-ab14-457e-bf8f-fdef4de083e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-62b91eb5-c52kj"
Oct 30 19:38:07.718375 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:07.718349    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-62b91eb5-c52kj"
Oct 30 19:38:07.945334 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:07.945258    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5016f41a-d492t"]
Oct 30 19:38:07.951833 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:07.951811    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5016f41a-d492t"
Oct 30 19:38:07.970154 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:07.970131    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5016f41a-d492t"]
Oct 30 19:38:08.029644 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:08.029603    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nj5cv\" (UniqueName: \"kubernetes.io/projected/ecf613ff-7fd9-462f-af0b-95256bf973f9-kube-api-access-nj5cv\") pod \"maas-api-revoke-keys-e2e-migrate-5016f41a-d492t\" (UID: \"ecf613ff-7fd9-462f-af0b-95256bf973f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5016f41a-d492t"
Oct 30 19:38:08.029993 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:08.029921    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ecf613ff-7fd9-462f-af0b-95256bf973f9-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5016f41a-d492t\" (UID: \"ecf613ff-7fd9-462f-af0b-95256bf973f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5016f41a-d492t"
Oct 30 19:38:08.030434 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:08.030022    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ecf613ff-7fd9-462f-af0b-95256bf973f9-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5016f41a-d492t\" (UID: \"ecf613ff-7fd9-462f-af0b-95256bf973f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5016f41a-d492t"
Oct 30 19:38:08.050923 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:38:08.036532    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcbde9d9_ab14_457e_bf8f_fdef4de083e0.slice/crio-860c34157e24a576fe669df58055e554be01b9dccdb24eee4d764b1d170ea0b0 WatchSource:0}: Error finding container 860c34157e24a576fe669df58055e554be01b9dccdb24eee4d764b1d170ea0b0: Status 404 returned error can't find the container with id 860c34157e24a576fe669df58055e554be01b9dccdb24eee4d764b1d170ea0b0
Oct 30 19:38:08.050923 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:08.040895    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-62b91eb5-c52kj"]
Oct 30 19:38:08.066533 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:08.066502    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-62b91eb5-c52kj" event={"ID":"dcbde9d9-ab14-457e-bf8f-fdef4de083e0","Type":"ContainerStarted","Data":"860c34157e24a576fe669df58055e554be01b9dccdb24eee4d764b1d170ea0b0"}
Oct 30 19:38:08.131150 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:08.131121    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ecf613ff-7fd9-462f-af0b-95256bf973f9-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5016f41a-d492t\" (UID: \"ecf613ff-7fd9-462f-af0b-95256bf973f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5016f41a-d492t"
Oct 30 19:38:08.131318 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:08.131297    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nj5cv\" (UniqueName: \"kubernetes.io/projected/ecf613ff-7fd9-462f-af0b-95256bf973f9-kube-api-access-nj5cv\") pod \"maas-api-revoke-keys-e2e-migrate-5016f41a-d492t\" (UID: \"ecf613ff-7fd9-462f-af0b-95256bf973f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5016f41a-d492t"
Oct 30 19:38:08.131675 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:08.131658    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ecf613ff-7fd9-462f-af0b-95256bf973f9-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5016f41a-d492t\" (UID: \"ecf613ff-7fd9-462f-af0b-95256bf973f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5016f41a-d492t"
Oct 30 19:38:08.132295 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:08.132272    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ecf613ff-7fd9-462f-af0b-95256bf973f9-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5016f41a-d492t\" (UID: \"ecf613ff-7fd9-462f-af0b-95256bf973f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5016f41a-d492t"
Oct 30 19:38:08.133787 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:08.133768    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ecf613ff-7fd9-462f-af0b-95256bf973f9-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5016f41a-d492t\" (UID: \"ecf613ff-7fd9-462f-af0b-95256bf973f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5016f41a-d492t"
Oct 30 19:38:08.154634 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:08.154604    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nj5cv\" (UniqueName: \"kubernetes.io/projected/ecf613ff-7fd9-462f-af0b-95256bf973f9-kube-api-access-nj5cv\") pod \"maas-api-revoke-keys-e2e-migrate-5016f41a-d492t\" (UID: \"ecf613ff-7fd9-462f-af0b-95256bf973f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5016f41a-d492t"
Oct 30 19:38:08.282886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:08.270363    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5016f41a-d492t"
Oct 30 19:38:09.088930 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:09.074268    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8bbfccd1-k7zbj"]
Oct 30 19:38:09.153885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:09.146800    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8bbfccd1-k7zbj"]
Oct 30 19:38:09.628422 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:09.628375    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-62b91eb5-c52kj" event={"ID":"dcbde9d9-ab14-457e-bf8f-fdef4de083e0","Type":"ContainerStarted","Data":"3658afbbef1aa4d00622f94040159a0d7c50079ccd6e553cb9bfce018567fe2f"}
Oct 30 19:38:09.839366 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:09.839303    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-62b91eb5-c52kj" podStartSLOduration=2.839284487 podStartE2EDuration="2.839284487s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:38:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:38:09.75436372 +0000 UTC m=+1001.597295670" watchObservedRunningTime="2026-10-30 19:38:09.839284487 +0000 UTC m=+1001.682216427"
Oct 30 19:38:09.849994 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:09.849950    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5016f41a-d492t"]
Oct 30 19:38:10.296633 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:10.296592    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c6a39cdf-82d1-44a3-902f-745528b5313b" path="/var/lib/kubelet/pods/c6a39cdf-82d1-44a3-902f-745528b5313b/volumes"
Oct 30 19:38:10.777538 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:10.774940    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5016f41a-d492t" event={"ID":"ecf613ff-7fd9-462f-af0b-95256bf973f9","Type":"ContainerStarted","Data":"e8f596f7faf0dcd479eb14d426ee9af3aa93b94b8544f151f06e442474cdb63f"}
Oct 30 19:38:10.777538 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:10.775038    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5016f41a-d492t" event={"ID":"ecf613ff-7fd9-462f-af0b-95256bf973f9","Type":"ContainerStarted","Data":"46c823be0b79301f1d40d3d1240eda9402366d30799eb18b83791e7d80cc8549"}
Oct 30 19:38:10.806269 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:10.805919    5907 generic.go:356] "Generic (PLEG): container finished" podID="dcbde9d9-ab14-457e-bf8f-fdef4de083e0" containerID="3658afbbef1aa4d00622f94040159a0d7c50079ccd6e553cb9bfce018567fe2f" exitCode=0
Oct 30 19:38:10.806269 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:10.806098    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-62b91eb5-c52kj" event={"ID":"dcbde9d9-ab14-457e-bf8f-fdef4de083e0","Type":"ContainerDied","Data":"3658afbbef1aa4d00622f94040159a0d7c50079ccd6e553cb9bfce018567fe2f"}
Oct 30 19:38:11.085590 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:11.085483    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-df776e5c-2nbtv"]
Oct 30 19:38:11.307886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:11.298329    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-df776e5c-2nbtv"
Oct 30 19:38:11.321972 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:11.321931    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-df776e5c-2nbtv"]
Oct 30 19:38:11.372196 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:11.372153    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cd697abb-3adc-4a4d-b30b-44fb8dea4b7f-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-derive-df776e5c-2nbtv\" (UID: \"cd697abb-3adc-4a4d-b30b-44fb8dea4b7f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-df776e5c-2nbtv"
Oct 30 19:38:11.372499 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:11.372468    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cd697abb-3adc-4a4d-b30b-44fb8dea4b7f-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-derive-df776e5c-2nbtv\" (UID: \"cd697abb-3adc-4a4d-b30b-44fb8dea4b7f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-df776e5c-2nbtv"
Oct 30 19:38:11.372728 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:11.372710    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t57hv\" (UniqueName: \"kubernetes.io/projected/cd697abb-3adc-4a4d-b30b-44fb8dea4b7f-kube-api-access-t57hv\") pod \"maas-api-revoke-keys-e2e-derive-df776e5c-2nbtv\" (UID: \"cd697abb-3adc-4a4d-b30b-44fb8dea4b7f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-df776e5c-2nbtv"
Oct 30 19:38:11.482042 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:11.482004    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cd697abb-3adc-4a4d-b30b-44fb8dea4b7f-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-derive-df776e5c-2nbtv\" (UID: \"cd697abb-3adc-4a4d-b30b-44fb8dea4b7f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-df776e5c-2nbtv"
Oct 30 19:38:11.482829 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:11.482804    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cd697abb-3adc-4a4d-b30b-44fb8dea4b7f-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-derive-df776e5c-2nbtv\" (UID: \"cd697abb-3adc-4a4d-b30b-44fb8dea4b7f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-df776e5c-2nbtv"
Oct 30 19:38:11.488413 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:11.488389    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t57hv\" (UniqueName: \"kubernetes.io/projected/cd697abb-3adc-4a4d-b30b-44fb8dea4b7f-kube-api-access-t57hv\") pod \"maas-api-revoke-keys-e2e-derive-df776e5c-2nbtv\" (UID: \"cd697abb-3adc-4a4d-b30b-44fb8dea4b7f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-df776e5c-2nbtv"
Oct 30 19:38:11.488967 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:11.483946    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cd697abb-3adc-4a4d-b30b-44fb8dea4b7f-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-derive-df776e5c-2nbtv\" (UID: \"cd697abb-3adc-4a4d-b30b-44fb8dea4b7f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-df776e5c-2nbtv"
Oct 30 19:38:11.495708 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:11.495651    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5016f41a-d492t" podStartSLOduration=4.495633663 podStartE2EDuration="4.495633663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:38:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:38:10.958206643 +0000 UTC m=+1002.801138584" watchObservedRunningTime="2026-10-30 19:38:11.495633663 +0000 UTC m=+1003.338565602"
Oct 30 19:38:11.574757 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:11.574722    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cd697abb-3adc-4a4d-b30b-44fb8dea4b7f-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-derive-df776e5c-2nbtv\" (UID: \"cd697abb-3adc-4a4d-b30b-44fb8dea4b7f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-df776e5c-2nbtv"
Oct 30 19:38:11.575750 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:11.575721    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t57hv\" (UniqueName: \"kubernetes.io/projected/cd697abb-3adc-4a4d-b30b-44fb8dea4b7f-kube-api-access-t57hv\") pod \"maas-api-revoke-keys-e2e-derive-df776e5c-2nbtv\" (UID: \"cd697abb-3adc-4a4d-b30b-44fb8dea4b7f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-df776e5c-2nbtv"
Oct 30 19:38:11.664948 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:11.664431    5907 patch_prober.go:29] interesting pod/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:38:11.664948 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:11.664501    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np" podUID="71285665-ae23-4783-a24f-535dccfb68c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:38:11.699885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:11.699052    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-df776e5c-2nbtv"
Oct 30 19:38:11.876732 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:11.873557    5907 generic.go:356] "Generic (PLEG): container finished" podID="ecf613ff-7fd9-462f-af0b-95256bf973f9" containerID="e8f596f7faf0dcd479eb14d426ee9af3aa93b94b8544f151f06e442474cdb63f" exitCode=0
Oct 30 19:38:11.876732 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:11.874139    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5016f41a-d492t" event={"ID":"ecf613ff-7fd9-462f-af0b-95256bf973f9","Type":"ContainerDied","Data":"e8f596f7faf0dcd479eb14d426ee9af3aa93b94b8544f151f06e442474cdb63f"}
Oct 30 19:38:12.343262 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:12.314474    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-df776e5c-2nbtv"]
Oct 30 19:38:12.358088 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:12.358056    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-62b91eb5-c52kj"
Oct 30 19:38:12.538884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:12.499635    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dcbde9d9-ab14-457e-bf8f-fdef4de083e0-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dcbde9d9-ab14-457e-bf8f-fdef4de083e0-maas-api-xks-ca\") pod \"dcbde9d9-ab14-457e-bf8f-fdef4de083e0\" (UID: \"dcbde9d9-ab14-457e-bf8f-fdef4de083e0\") "
Oct 30 19:38:12.556276 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:12.556182    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dcbde9d9-ab14-457e-bf8f-fdef4de083e0-kube-api-access-fs6vx\" (UniqueName: \"kubernetes.io/projected/dcbde9d9-ab14-457e-bf8f-fdef4de083e0-kube-api-access-fs6vx\") pod \"dcbde9d9-ab14-457e-bf8f-fdef4de083e0\" (UID: \"dcbde9d9-ab14-457e-bf8f-fdef4de083e0\") "
Oct 30 19:38:12.556653 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:12.556631    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dcbde9d9-ab14-457e-bf8f-fdef4de083e0-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dcbde9d9-ab14-457e-bf8f-fdef4de083e0-maas-api-openshift-ca\") pod \"dcbde9d9-ab14-457e-bf8f-fdef4de083e0\" (UID: \"dcbde9d9-ab14-457e-bf8f-fdef4de083e0\") "
Oct 30 19:38:12.558651 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:12.558615    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dcbde9d9-ab14-457e-bf8f-fdef4de083e0-maas-api-openshift-ca" pod "dcbde9d9-ab14-457e-bf8f-fdef4de083e0" (UID: "dcbde9d9-ab14-457e-bf8f-fdef4de083e0"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:38:12.600024 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:12.598992    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dcbde9d9-ab14-457e-bf8f-fdef4de083e0-maas-api-xks-ca" pod "dcbde9d9-ab14-457e-bf8f-fdef4de083e0" (UID: "dcbde9d9-ab14-457e-bf8f-fdef4de083e0"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:38:12.663445 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:12.663021    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcbde9d9-ab14-457e-bf8f-fdef4de083e0-kube-api-access-fs6vx" pod "dcbde9d9-ab14-457e-bf8f-fdef4de083e0" (UID: "dcbde9d9-ab14-457e-bf8f-fdef4de083e0"). InnerVolumeSpecName "kube-api-access-fs6vx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:12.670685 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:12.666218    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fs6vx\" (UniqueName: \"kubernetes.io/projected/dcbde9d9-ab14-457e-bf8f-fdef4de083e0-kube-api-access-fs6vx\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:12.670685 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:12.666253    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dcbde9d9-ab14-457e-bf8f-fdef4de083e0-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:12.670685 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:12.666271    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dcbde9d9-ab14-457e-bf8f-fdef4de083e0-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:12.885339 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:12.885291    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-df776e5c-2nbtv" event={"ID":"cd697abb-3adc-4a4d-b30b-44fb8dea4b7f","Type":"ContainerStarted","Data":"31234e6add94ce0b0f36018e2670e43f4999496bbc48af2ab53ec818529efb34"}
Oct 30 19:38:12.887015 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:12.886985    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-62b91eb5-c52kj"
Oct 30 19:38:12.887015 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:12.887011    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-62b91eb5-c52kj" event={"ID":"dcbde9d9-ab14-457e-bf8f-fdef4de083e0","Type":"ContainerDied","Data":"860c34157e24a576fe669df58055e554be01b9dccdb24eee4d764b1d170ea0b0"}
Oct 30 19:38:12.887222 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:12.887047    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="860c34157e24a576fe669df58055e554be01b9dccdb24eee4d764b1d170ea0b0"
Oct 30 19:38:13.183800 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:13.183770    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5016f41a-d492t"
Oct 30 19:38:13.282378 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:13.282334    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ecf613ff-7fd9-462f-af0b-95256bf973f9-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ecf613ff-7fd9-462f-af0b-95256bf973f9-maas-api-xks-ca\") pod \"ecf613ff-7fd9-462f-af0b-95256bf973f9\" (UID: \"ecf613ff-7fd9-462f-af0b-95256bf973f9\") "
Oct 30 19:38:13.282564 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:13.282423    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ecf613ff-7fd9-462f-af0b-95256bf973f9-kube-api-access-nj5cv\" (UniqueName: \"kubernetes.io/projected/ecf613ff-7fd9-462f-af0b-95256bf973f9-kube-api-access-nj5cv\") pod \"ecf613ff-7fd9-462f-af0b-95256bf973f9\" (UID: \"ecf613ff-7fd9-462f-af0b-95256bf973f9\") "
Oct 30 19:38:13.282564 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:13.282543    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ecf613ff-7fd9-462f-af0b-95256bf973f9-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ecf613ff-7fd9-462f-af0b-95256bf973f9-maas-api-openshift-ca\") pod \"ecf613ff-7fd9-462f-af0b-95256bf973f9\" (UID: \"ecf613ff-7fd9-462f-af0b-95256bf973f9\") "
Oct 30 19:38:13.284388 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:13.284362    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecf613ff-7fd9-462f-af0b-95256bf973f9-maas-api-openshift-ca" pod "ecf613ff-7fd9-462f-af0b-95256bf973f9" (UID: "ecf613ff-7fd9-462f-af0b-95256bf973f9"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:38:13.286270 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:13.286236    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecf613ff-7fd9-462f-af0b-95256bf973f9-kube-api-access-nj5cv" pod "ecf613ff-7fd9-462f-af0b-95256bf973f9" (UID: "ecf613ff-7fd9-462f-af0b-95256bf973f9"). InnerVolumeSpecName "kube-api-access-nj5cv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:13.286375 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:13.286260    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecf613ff-7fd9-462f-af0b-95256bf973f9-maas-api-xks-ca" pod "ecf613ff-7fd9-462f-af0b-95256bf973f9" (UID: "ecf613ff-7fd9-462f-af0b-95256bf973f9"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:38:13.386529 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:13.386439    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ecf613ff-7fd9-462f-af0b-95256bf973f9-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:13.386529 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:13.386480    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nj5cv\" (UniqueName: \"kubernetes.io/projected/ecf613ff-7fd9-462f-af0b-95256bf973f9-kube-api-access-nj5cv\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:13.386529 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:13.386496    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ecf613ff-7fd9-462f-af0b-95256bf973f9-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:13.933477 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:13.933424    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5016f41a-d492t" event={"ID":"ecf613ff-7fd9-462f-af0b-95256bf973f9","Type":"ContainerDied","Data":"46c823be0b79301f1d40d3d1240eda9402366d30799eb18b83791e7d80cc8549"}
Oct 30 19:38:13.937904 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:13.936517    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5016f41a-d492t"
Oct 30 19:38:13.937904 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:13.937113    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="46c823be0b79301f1d40d3d1240eda9402366d30799eb18b83791e7d80cc8549"
Oct 30 19:38:13.939380 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:13.939353    5907 generic.go:356] "Generic (PLEG): container finished" podID="cd697abb-3adc-4a4d-b30b-44fb8dea4b7f" containerID="319f41f4faf68b13cc46cee89239314f8e9e92b66ff427cc50f82d4a8df1be80" exitCode=0
Oct 30 19:38:13.939685 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:13.939667    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-df776e5c-2nbtv" event={"ID":"cd697abb-3adc-4a4d-b30b-44fb8dea4b7f","Type":"ContainerDied","Data":"319f41f4faf68b13cc46cee89239314f8e9e92b66ff427cc50f82d4a8df1be80"}
Oct 30 19:38:15.187326 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:15.187295    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-df776e5c-2nbtv"
Oct 30 19:38:15.243013 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:15.242925    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cd697abb-3adc-4a4d-b30b-44fb8dea4b7f-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cd697abb-3adc-4a4d-b30b-44fb8dea4b7f-maas-api-xks-ca\") pod \"cd697abb-3adc-4a4d-b30b-44fb8dea4b7f\" (UID: \"cd697abb-3adc-4a4d-b30b-44fb8dea4b7f\") "
Oct 30 19:38:15.243209 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:15.243118    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cd697abb-3adc-4a4d-b30b-44fb8dea4b7f-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cd697abb-3adc-4a4d-b30b-44fb8dea4b7f-maas-api-openshift-ca\") pod \"cd697abb-3adc-4a4d-b30b-44fb8dea4b7f\" (UID: \"cd697abb-3adc-4a4d-b30b-44fb8dea4b7f\") "
Oct 30 19:38:15.243209 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:15.243150    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd697abb-3adc-4a4d-b30b-44fb8dea4b7f-kube-api-access-t57hv\" (UniqueName: \"kubernetes.io/projected/cd697abb-3adc-4a4d-b30b-44fb8dea4b7f-kube-api-access-t57hv\") pod \"cd697abb-3adc-4a4d-b30b-44fb8dea4b7f\" (UID: \"cd697abb-3adc-4a4d-b30b-44fb8dea4b7f\") "
Oct 30 19:38:15.243702 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:15.243468    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd697abb-3adc-4a4d-b30b-44fb8dea4b7f-maas-api-openshift-ca" pod "cd697abb-3adc-4a4d-b30b-44fb8dea4b7f" (UID: "cd697abb-3adc-4a4d-b30b-44fb8dea4b7f"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:38:15.244504 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:15.244476    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cd697abb-3adc-4a4d-b30b-44fb8dea4b7f-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:15.245932 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:15.245903    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd697abb-3adc-4a4d-b30b-44fb8dea4b7f-maas-api-xks-ca" pod "cd697abb-3adc-4a4d-b30b-44fb8dea4b7f" (UID: "cd697abb-3adc-4a4d-b30b-44fb8dea4b7f"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:38:15.246083 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:15.246041    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd697abb-3adc-4a4d-b30b-44fb8dea4b7f-kube-api-access-t57hv" pod "cd697abb-3adc-4a4d-b30b-44fb8dea4b7f" (UID: "cd697abb-3adc-4a4d-b30b-44fb8dea4b7f"). InnerVolumeSpecName "kube-api-access-t57hv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:15.345901 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:15.345844    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cd697abb-3adc-4a4d-b30b-44fb8dea4b7f-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:15.345901 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:15.345887    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t57hv\" (UniqueName: \"kubernetes.io/projected/cd697abb-3adc-4a4d-b30b-44fb8dea4b7f-kube-api-access-t57hv\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:15.960206 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:15.960175    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-df776e5c-2nbtv"
Oct 30 19:38:15.960393 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:15.960203    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-df776e5c-2nbtv" event={"ID":"cd697abb-3adc-4a4d-b30b-44fb8dea4b7f","Type":"ContainerDied","Data":"31234e6add94ce0b0f36018e2670e43f4999496bbc48af2ab53ec818529efb34"}
Oct 30 19:38:15.960393 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:15.960249    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="31234e6add94ce0b0f36018e2670e43f4999496bbc48af2ab53ec818529efb34"
Oct 30 19:38:17.045579 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:17.045535    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d23bf141-d6lxg"]
Oct 30 19:38:17.062317 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:17.062280    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d23bf141-d6lxg"]
Oct 30 19:38:17.069338 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:17.069310    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a566702d-jznrx"]
Oct 30 19:38:17.075751 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:17.075719    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a566702d-jznrx"]
Oct 30 19:38:17.510317 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:38:17.510276    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:38:17.849982 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:17.849807    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-62b91eb5-64bb64b574-5rt9l"]
Oct 30 19:38:17.860146 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:17.860113    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-62b91eb5-6579c7c78c-gtv77"]
Oct 30 19:38:18.031931 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:18.031887    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f74b9672-t56vz"]
Oct 30 19:38:18.038622 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:18.038590    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f74b9672-t56vz"]
Oct 30 19:38:18.298930 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:18.298896    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4c779354-fa02-4c26-999a-bd75c8a3e351" path="/var/lib/kubelet/pods/4c779354-fa02-4c26-999a-bd75c8a3e351/volumes"
Oct 30 19:38:18.299607 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:18.299573    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6d1935cb-a979-4aad-aa3f-988c94abd3d6" path="/var/lib/kubelet/pods/6d1935cb-a979-4aad-aa3f-988c94abd3d6/volumes"
Oct 30 19:38:18.300193 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:18.300169    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d4da88bf-a2b6-4055-8d91-50f0d10decdd" path="/var/lib/kubelet/pods/d4da88bf-a2b6-4055-8d91-50f0d10decdd/volumes"
Oct 30 19:38:18.611991 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:18.611954    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-5016f41a-bff455988-57j6h"]
Oct 30 19:38:18.632461 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:18.632429    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-5016f41a-bd5df848c-mvxr6"]
Oct 30 19:38:21.121771 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:21.121157    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-df776e5c-6b7db86b98-xbxfg"]
Oct 30 19:38:21.146708 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:21.146671    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-df776e5c-57c7bd5f5f-rvsvz"]
Oct 30 19:38:22.850783 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:22.850742    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-62b91eb5-64bb64b574-5rt9l" podUID="4edc707a-3196-48f1-919a-98e5b12a234e" containerName="payload-processing" containerID="cri-o://5e2002aa9434762a3dc4dfe1007ef3a05b6de1a2941f58d0e4871620a5b94b08" gracePeriod=25
Oct 30 19:38:22.861332 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:22.861300    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-62b91eb5-6579c7c78c-gtv77" podUID="2645bd0d-99ac-4e8d-a136-f3d37545d517" containerName="payload-pre-processing" containerID="cri-o://fa3a5cd6e1b5f2729c5c745561f31c9fa505725feab02444b601decf40ba23ef" gracePeriod=25
Oct 30 19:38:23.263950 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:23.263924    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-62b91eb5-64bb64b574-5rt9l"
Oct 30 19:38:23.286772 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:23.286749    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-62b91eb5-6579c7c78c-gtv77"
Oct 30 19:38:23.486941 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:23.480306    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2645bd0d-99ac-4e8d-a136-f3d37545d517-kube-api-access-fk67m\" (UniqueName: \"kubernetes.io/projected/2645bd0d-99ac-4e8d-a136-f3d37545d517-kube-api-access-fk67m\") pod \"2645bd0d-99ac-4e8d-a136-f3d37545d517\" (UID: \"2645bd0d-99ac-4e8d-a136-f3d37545d517\") "
Oct 30 19:38:23.486941 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:23.480503    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2645bd0d-99ac-4e8d-a136-f3d37545d517-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2645bd0d-99ac-4e8d-a136-f3d37545d517-plugins-config-volume\") pod \"2645bd0d-99ac-4e8d-a136-f3d37545d517\" (UID: \"2645bd0d-99ac-4e8d-a136-f3d37545d517\") "
Oct 30 19:38:23.486941 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:23.480723    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4edc707a-3196-48f1-919a-98e5b12a234e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4edc707a-3196-48f1-919a-98e5b12a234e-plugins-config-volume\") pod \"4edc707a-3196-48f1-919a-98e5b12a234e\" (UID: \"4edc707a-3196-48f1-919a-98e5b12a234e\") "
Oct 30 19:38:23.486941 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:23.480901    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4edc707a-3196-48f1-919a-98e5b12a234e-kube-api-access-9cthl\" (UniqueName: \"kubernetes.io/projected/4edc707a-3196-48f1-919a-98e5b12a234e-kube-api-access-9cthl\") pod \"4edc707a-3196-48f1-919a-98e5b12a234e\" (UID: \"4edc707a-3196-48f1-919a-98e5b12a234e\") "
Oct 30 19:38:23.526611 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:23.526541    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4edc707a-3196-48f1-919a-98e5b12a234e-plugins-config-volume" pod "4edc707a-3196-48f1-919a-98e5b12a234e" (UID: "4edc707a-3196-48f1-919a-98e5b12a234e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:38:23.543296 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:23.542810    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2645bd0d-99ac-4e8d-a136-f3d37545d517-plugins-config-volume" pod "2645bd0d-99ac-4e8d-a136-f3d37545d517" (UID: "2645bd0d-99ac-4e8d-a136-f3d37545d517"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:38:23.591633 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:23.591581    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2645bd0d-99ac-4e8d-a136-f3d37545d517-kube-api-access-fk67m" pod "2645bd0d-99ac-4e8d-a136-f3d37545d517" (UID: "2645bd0d-99ac-4e8d-a136-f3d37545d517"). InnerVolumeSpecName "kube-api-access-fk67m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:23.613511 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:23.612666    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fk67m\" (UniqueName: \"kubernetes.io/projected/2645bd0d-99ac-4e8d-a136-f3d37545d517-kube-api-access-fk67m\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:23.613511 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:23.612709    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2645bd0d-99ac-4e8d-a136-f3d37545d517-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:23.613511 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:23.612725    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4edc707a-3196-48f1-919a-98e5b12a234e-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:23.639895 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:23.621400    5907 generic.go:356] "Generic (PLEG): container finished" podID="ce9284b5-c1c6-4390-a5aa-410c7538e9eb" containerID="42591b20effe399ad712e14ff8d7ce6d7a59128a3ff52a71e3cc807d7904b742" exitCode=0
Oct 30 19:38:23.639895 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:23.621682    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh" event={"ID":"ce9284b5-c1c6-4390-a5aa-410c7538e9eb","Type":"ContainerDied","Data":"42591b20effe399ad712e14ff8d7ce6d7a59128a3ff52a71e3cc807d7904b742"}
Oct 30 19:38:23.639895 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:23.622258    5907 scope.go:122] "RemoveContainer" containerID="42591b20effe399ad712e14ff8d7ce6d7a59128a3ff52a71e3cc807d7904b742"
Oct 30 19:38:23.666089 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:23.642956    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-5016f41a-bff455988-57j6h" podUID="a3c0d05d-67f1-4e07-82d0-516ff973b2f2" containerName="payload-processing" containerID="cri-o://520e7ae058f45d40c48f1c38ad09df60e8743b57f129a229e4e5ae3c0d1e27cb" gracePeriod=25
Oct 30 19:38:23.666089 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:23.644113    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5016f41a-bd5df848c-mvxr6" podUID="7659a68f-a908-46c0-a61c-74f827b84339" containerName="payload-pre-processing" containerID="cri-o://b09bd77b046c684d36c7ea545c0821bbd163558ed6b3f16b47b6b25a5b04be29" gracePeriod=25
Oct 30 19:38:23.666089 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:23.649620    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:38:23.666089 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:23.649717    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:38:23.666089 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:23.649764    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:38:23.937149 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:23.854026    5907 generic.go:356] "Generic (PLEG): container finished" podID="2645bd0d-99ac-4e8d-a136-f3d37545d517" containerID="fa3a5cd6e1b5f2729c5c745561f31c9fa505725feab02444b601decf40ba23ef" exitCode=0
Oct 30 19:38:23.937149 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:23.854216    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-62b91eb5-6579c7c78c-gtv77" event={"ID":"2645bd0d-99ac-4e8d-a136-f3d37545d517","Type":"ContainerDied","Data":"fa3a5cd6e1b5f2729c5c745561f31c9fa505725feab02444b601decf40ba23ef"}
Oct 30 19:38:23.937149 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:23.854271    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-62b91eb5-6579c7c78c-gtv77" event={"ID":"2645bd0d-99ac-4e8d-a136-f3d37545d517","Type":"ContainerDied","Data":"cf097438beaadaaaa361bec5afb5fb86d3a8f41c9ede50aa7f45da64d5d1107c"}
Oct 30 19:38:23.937149 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:23.854299    5907 scope.go:122] "RemoveContainer" containerID="fa3a5cd6e1b5f2729c5c745561f31c9fa505725feab02444b601decf40ba23ef"
Oct 30 19:38:23.937149 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:23.854458    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-62b91eb5-6579c7c78c-gtv77"
Oct 30 19:38:24.172014 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:24.127095    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4edc707a-3196-48f1-919a-98e5b12a234e-kube-api-access-9cthl" pod "4edc707a-3196-48f1-919a-98e5b12a234e" (UID: "4edc707a-3196-48f1-919a-98e5b12a234e"). InnerVolumeSpecName "kube-api-access-9cthl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:24.172014 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:24.157277    5907 generic.go:356] "Generic (PLEG): container finished" podID="4edc707a-3196-48f1-919a-98e5b12a234e" containerID="5e2002aa9434762a3dc4dfe1007ef3a05b6de1a2941f58d0e4871620a5b94b08" exitCode=0
Oct 30 19:38:24.172014 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:24.157478    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-62b91eb5-64bb64b574-5rt9l" event={"ID":"4edc707a-3196-48f1-919a-98e5b12a234e","Type":"ContainerDied","Data":"5e2002aa9434762a3dc4dfe1007ef3a05b6de1a2941f58d0e4871620a5b94b08"}
Oct 30 19:38:24.172014 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:24.157533    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-62b91eb5-64bb64b574-5rt9l" event={"ID":"4edc707a-3196-48f1-919a-98e5b12a234e","Type":"ContainerDied","Data":"229b7e52a920f4842351358ee2d698cd591dd7224be21998bc324e0751830960"}
Oct 30 19:38:24.172014 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:24.157609    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-62b91eb5-64bb64b574-5rt9l"
Oct 30 19:38:24.182072 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:24.182027    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9cthl\" (UniqueName: \"kubernetes.io/projected/4edc707a-3196-48f1-919a-98e5b12a234e-kube-api-access-9cthl\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:24.440177 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:24.440058    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:38:24.503095 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:24.469091    5907 scope.go:122] "RemoveContainer" containerID="fa3a5cd6e1b5f2729c5c745561f31c9fa505725feab02444b601decf40ba23ef"
Oct 30 19:38:24.679940 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:38:24.590040    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa3a5cd6e1b5f2729c5c745561f31c9fa505725feab02444b601decf40ba23ef\": container with ID starting with fa3a5cd6e1b5f2729c5c745561f31c9fa505725feab02444b601decf40ba23ef not found: ID does not exist" containerID="fa3a5cd6e1b5f2729c5c745561f31c9fa505725feab02444b601decf40ba23ef"
Oct 30 19:38:24.679940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:24.590092    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa3a5cd6e1b5f2729c5c745561f31c9fa505725feab02444b601decf40ba23ef"} err="failed to get container status \"fa3a5cd6e1b5f2729c5c745561f31c9fa505725feab02444b601decf40ba23ef\": rpc error: code = NotFound desc = could not find container \"fa3a5cd6e1b5f2729c5c745561f31c9fa505725feab02444b601decf40ba23ef\": container with ID starting with fa3a5cd6e1b5f2729c5c745561f31c9fa505725feab02444b601decf40ba23ef not found: ID does not exist"
Oct 30 19:38:24.679940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:24.590120    5907 scope.go:122] "RemoveContainer" containerID="5e2002aa9434762a3dc4dfe1007ef3a05b6de1a2941f58d0e4871620a5b94b08"
Oct 30 19:38:25.334032 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:25.333947    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dbc21c9e-rlz2s"]
Oct 30 19:38:25.394554 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:25.358936    5907 scope.go:122] "RemoveContainer" containerID="5e2002aa9434762a3dc4dfe1007ef3a05b6de1a2941f58d0e4871620a5b94b08"
Oct 30 19:38:25.407095 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:38:25.387852    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e2002aa9434762a3dc4dfe1007ef3a05b6de1a2941f58d0e4871620a5b94b08\": container with ID starting with 5e2002aa9434762a3dc4dfe1007ef3a05b6de1a2941f58d0e4871620a5b94b08 not found: ID does not exist" containerID="5e2002aa9434762a3dc4dfe1007ef3a05b6de1a2941f58d0e4871620a5b94b08"
Oct 30 19:38:25.407095 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:25.405767    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e2002aa9434762a3dc4dfe1007ef3a05b6de1a2941f58d0e4871620a5b94b08"} err="failed to get container status \"5e2002aa9434762a3dc4dfe1007ef3a05b6de1a2941f58d0e4871620a5b94b08\": rpc error: code = NotFound desc = could not find container \"5e2002aa9434762a3dc4dfe1007ef3a05b6de1a2941f58d0e4871620a5b94b08\": container with ID starting with 5e2002aa9434762a3dc4dfe1007ef3a05b6de1a2941f58d0e4871620a5b94b08 not found: ID does not exist"
Oct 30 19:38:25.411885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:25.409943    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dbc21c9e-rlz2s"]
Oct 30 19:38:25.453476 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:25.435615    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-62b91eb5-c52kj"]
Oct 30 19:38:25.453476 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:25.444556    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-62b91eb5-c52kj"]
Oct 30 19:38:25.468974 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:25.464850    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"]
Oct 30 19:38:25.469441 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:25.469389    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz" podUID="a28e0e27-b1cb-4d23-a869-a291ccfd30ae" containerName="istio-proxy" containerID="cri-o://1070687e67289e59a279dbcf53d3a636424a458821396b18052231793cb79091" gracePeriod=30
Oct 30 19:38:25.478906 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:25.473354    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-62b91eb5-6579c7c78c-gtv77"]
Oct 30 19:38:25.495976 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:25.489363    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-62b91eb5-6579c7c78c-gtv77"]
Oct 30 19:38:25.500102 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:25.500054    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-62b91eb5-64bb64b574-5rt9l"]
Oct 30 19:38:25.527659 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:25.516983    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-62b91eb5-64bb64b574-5rt9l"]
Oct 30 19:38:26.173884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.170246    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-df776e5c-6b7db86b98-xbxfg" podUID="207b8150-f938-4bce-a4d6-473fdf54212d" containerName="payload-processing" containerID="cri-o://209f228f2e91ea65e264665577cd1fa7e92de54f272c298e6d1f39b9bfbed352" gracePeriod=25
Oct 30 19:38:26.266934 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.223253    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-df776e5c-57c7bd5f5f-rvsvz" podUID="f2c3dee9-2c52-44a2-a64e-aa661cce2452" containerName="payload-pre-processing" containerID="cri-o://0c14c210a21d68514911ce66e7caa7132f66fbf30f76c2e7c62834fd9e369e9d" gracePeriod=25
Oct 30 19:38:26.931664 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.931629    5907 generic.go:356] "Generic (PLEG): container finished" podID="a3c0d05d-67f1-4e07-82d0-516ff973b2f2" containerID="520e7ae058f45d40c48f1c38ad09df60e8743b57f129a229e4e5ae3c0d1e27cb" exitCode=0
Oct 30 19:38:26.947583 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.945450    5907 generic.go:356] "Generic (PLEG): container finished" podID="7659a68f-a908-46c0-a61c-74f827b84339" containerID="b09bd77b046c684d36c7ea545c0821bbd163558ed6b3f16b47b6b25a5b04be29" exitCode=0
Oct 30 19:38:26.976983 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.976948    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-5016f41a-bff455988-57j6h"
Oct 30 19:38:26.979600 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.979513    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5016f41a-bd5df848c-mvxr6"
Oct 30 19:38:26.980625 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.980578    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2645bd0d-99ac-4e8d-a136-f3d37545d517" path="/var/lib/kubelet/pods/2645bd0d-99ac-4e8d-a136-f3d37545d517/volumes"
Oct 30 19:38:26.981529 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.981504    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3638aed3-bb6e-479c-abc8-cc021bd35ca1" path="/var/lib/kubelet/pods/3638aed3-bb6e-479c-abc8-cc021bd35ca1/volumes"
Oct 30 19:38:26.982185 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.982148    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4edc707a-3196-48f1-919a-98e5b12a234e" path="/var/lib/kubelet/pods/4edc707a-3196-48f1-919a-98e5b12a234e/volumes"
Oct 30 19:38:26.983328 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.983280    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dcbde9d9-ab14-457e-bf8f-fdef4de083e0" path="/var/lib/kubelet/pods/dcbde9d9-ab14-457e-bf8f-fdef4de083e0/volumes"
Oct 30 19:38:26.983984 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.983962    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"]
Oct 30 19:38:26.985724 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.985701    5907 cpu_manager.go:414] "containerMap: removing container" podUID="cd697abb-3adc-4a4d-b30b-44fb8dea4b7f" containerName="revoke-keys"
Oct 30 19:38:26.985832 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.985744    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cd697abb-3adc-4a4d-b30b-44fb8dea4b7f" containerName="revoke-keys"
Oct 30 19:38:26.985832 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.985794    5907 cpu_manager.go:414] "containerMap: removing container" podUID="dcbde9d9-ab14-457e-bf8f-fdef4de083e0" containerName="revoke-keys"
Oct 30 19:38:26.985832 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.985820    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dcbde9d9-ab14-457e-bf8f-fdef4de083e0" containerName="revoke-keys"
Oct 30 19:38:26.986032 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.985852    5907 cpu_manager.go:414] "containerMap: removing container" podUID="a3c0d05d-67f1-4e07-82d0-516ff973b2f2" containerName="payload-processing"
Oct 30 19:38:26.986032 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.985902    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a3c0d05d-67f1-4e07-82d0-516ff973b2f2" containerName="payload-processing"
Oct 30 19:38:26.986032 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.985939    5907 cpu_manager.go:414] "containerMap: removing container" podUID="2645bd0d-99ac-4e8d-a136-f3d37545d517" containerName="payload-pre-processing"
Oct 30 19:38:26.986032 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.985948    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2645bd0d-99ac-4e8d-a136-f3d37545d517" containerName="payload-pre-processing"
Oct 30 19:38:26.986032 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.985988    5907 cpu_manager.go:414] "containerMap: removing container" podUID="ecf613ff-7fd9-462f-af0b-95256bf973f9" containerName="revoke-keys"
Oct 30 19:38:26.986032 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.985996    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ecf613ff-7fd9-462f-af0b-95256bf973f9" containerName="revoke-keys"
Oct 30 19:38:26.986032 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.986019    5907 cpu_manager.go:414] "containerMap: removing container" podUID="4edc707a-3196-48f1-919a-98e5b12a234e" containerName="payload-processing"
Oct 30 19:38:26.986366 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.986026    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4edc707a-3196-48f1-919a-98e5b12a234e" containerName="payload-processing"
Oct 30 19:38:26.986366 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.986059    5907 cpu_manager.go:414] "containerMap: removing container" podUID="7659a68f-a908-46c0-a61c-74f827b84339" containerName="payload-pre-processing"
Oct 30 19:38:26.986366 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.986067    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7659a68f-a908-46c0-a61c-74f827b84339" containerName="payload-pre-processing"
Oct 30 19:38:26.986366 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.986327    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="2645bd0d-99ac-4e8d-a136-f3d37545d517" containerName="payload-pre-processing"
Oct 30 19:38:26.986366 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.986344    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="ecf613ff-7fd9-462f-af0b-95256bf973f9" containerName="revoke-keys"
Oct 30 19:38:26.986973 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.986377    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="4edc707a-3196-48f1-919a-98e5b12a234e" containerName="payload-processing"
Oct 30 19:38:26.986973 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.986391    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="7659a68f-a908-46c0-a61c-74f827b84339" containerName="payload-pre-processing"
Oct 30 19:38:26.986973 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.986404    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="cd697abb-3adc-4a4d-b30b-44fb8dea4b7f" containerName="revoke-keys"
Oct 30 19:38:26.986973 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.986420    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="dcbde9d9-ab14-457e-bf8f-fdef4de083e0" containerName="revoke-keys"
Oct 30 19:38:26.986973 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.986452    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="a3c0d05d-67f1-4e07-82d0-516ff973b2f2" containerName="payload-processing"
Oct 30 19:38:26.992101 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.992083    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"]
Oct 30 19:38:26.992206 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.992125    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh" event={"ID":"ce9284b5-c1c6-4390-a5aa-410c7538e9eb","Type":"ContainerStarted","Data":"725d58d2a861352576472103d26ce61f093b5ab05c768b1d41bf8aacd7311c9c"}
Oct 30 19:38:26.992206 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.992158    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-5016f41a-bff455988-57j6h" event={"ID":"a3c0d05d-67f1-4e07-82d0-516ff973b2f2","Type":"ContainerDied","Data":"520e7ae058f45d40c48f1c38ad09df60e8743b57f129a229e4e5ae3c0d1e27cb"}
Oct 30 19:38:26.992206 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.992188    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5016f41a-bd5df848c-mvxr6" event={"ID":"7659a68f-a908-46c0-a61c-74f827b84339","Type":"ContainerDied","Data":"b09bd77b046c684d36c7ea545c0821bbd163558ed6b3f16b47b6b25a5b04be29"}
Oct 30 19:38:26.992369 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.992224    5907 scope.go:122] "RemoveContainer" containerID="520e7ae058f45d40c48f1c38ad09df60e8743b57f129a229e4e5ae3c0d1e27cb"
Oct 30 19:38:26.992425 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:26.992415    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"
Oct 30 19:38:27.015538 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.015005    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-b9c43c35-openshift-default-dockercfg-m6x68\""
Oct 30 19:38:27.047975 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.047913    5907 scope.go:122] "RemoveContainer" containerID="b09bd77b046c684d36c7ea545c0821bbd163558ed6b3f16b47b6b25a5b04be29"
Oct 30 19:38:27.048523 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.048499    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7659a68f-a908-46c0-a61c-74f827b84339-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7659a68f-a908-46c0-a61c-74f827b84339-plugins-config-volume\") pod \"7659a68f-a908-46c0-a61c-74f827b84339\" (UID: \"7659a68f-a908-46c0-a61c-74f827b84339\") "
Oct 30 19:38:27.048663 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.048645    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a3c0d05d-67f1-4e07-82d0-516ff973b2f2-kube-api-access-h6flk\" (UniqueName: \"kubernetes.io/projected/a3c0d05d-67f1-4e07-82d0-516ff973b2f2-kube-api-access-h6flk\") pod \"a3c0d05d-67f1-4e07-82d0-516ff973b2f2\" (UID: \"a3c0d05d-67f1-4e07-82d0-516ff973b2f2\") "
Oct 30 19:38:27.048786 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.048771    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7659a68f-a908-46c0-a61c-74f827b84339-kube-api-access-cdbr5\" (UniqueName: \"kubernetes.io/projected/7659a68f-a908-46c0-a61c-74f827b84339-kube-api-access-cdbr5\") pod \"7659a68f-a908-46c0-a61c-74f827b84339\" (UID: \"7659a68f-a908-46c0-a61c-74f827b84339\") "
Oct 30 19:38:27.049022 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.049004    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a3c0d05d-67f1-4e07-82d0-516ff973b2f2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3c0d05d-67f1-4e07-82d0-516ff973b2f2-plugins-config-volume\") pod \"a3c0d05d-67f1-4e07-82d0-516ff973b2f2\" (UID: \"a3c0d05d-67f1-4e07-82d0-516ff973b2f2\") "
Oct 30 19:38:27.049384 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.049361    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8399c42-686d-4164-b96d-e61f6a99c8bd-istio-podinfo\") pod \"e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") " pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"
Oct 30 19:38:27.049493 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.049477    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8399c42-686d-4164-b96d-e61f6a99c8bd-workload-socket\") pod \"e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") " pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"
Oct 30 19:38:27.049767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.049751    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8399c42-686d-4164-b96d-e61f6a99c8bd-istio-token\") pod \"e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") " pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"
Oct 30 19:38:27.049909 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.049892    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8399c42-686d-4164-b96d-e61f6a99c8bd-istio-envoy\") pod \"e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") " pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"
Oct 30 19:38:27.049981 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.049962    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8399c42-686d-4164-b96d-e61f6a99c8bd-credential-socket\") pod \"e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") " pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"
Oct 30 19:38:27.050081 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.049989    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8ldg\" (UniqueName: \"kubernetes.io/projected/c8399c42-686d-4164-b96d-e61f6a99c8bd-kube-api-access-n8ldg\") pod \"e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") " pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"
Oct 30 19:38:27.050138 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.050087    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8399c42-686d-4164-b96d-e61f6a99c8bd-istiod-ca-cert\") pod \"e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") " pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"
Oct 30 19:38:27.050138 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.050111    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8399c42-686d-4164-b96d-e61f6a99c8bd-istio-data\") pod \"e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") " pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"
Oct 30 19:38:27.050231 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.050188    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8399c42-686d-4164-b96d-e61f6a99c8bd-workload-certs\") pod \"e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") " pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"
Oct 30 19:38:27.052797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.052579    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3c0d05d-67f1-4e07-82d0-516ff973b2f2-plugins-config-volume" pod "a3c0d05d-67f1-4e07-82d0-516ff973b2f2" (UID: "a3c0d05d-67f1-4e07-82d0-516ff973b2f2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:38:27.073286 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.073229    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7659a68f-a908-46c0-a61c-74f827b84339-plugins-config-volume" pod "7659a68f-a908-46c0-a61c-74f827b84339" (UID: "7659a68f-a908-46c0-a61c-74f827b84339"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:38:27.088583 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.088452    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-51c63168-vkmwl"]
Oct 30 19:38:27.125951 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.123582    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-51c63168-vkmwl"]
Oct 30 19:38:27.157295 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.155373    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8399c42-686d-4164-b96d-e61f6a99c8bd-credential-socket\") pod \"e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") " pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"
Oct 30 19:38:27.157295 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.155426    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8ldg\" (UniqueName: \"kubernetes.io/projected/c8399c42-686d-4164-b96d-e61f6a99c8bd-kube-api-access-n8ldg\") pod \"e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") " pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"
Oct 30 19:38:27.157295 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.155514    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8399c42-686d-4164-b96d-e61f6a99c8bd-istiod-ca-cert\") pod \"e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") " pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"
Oct 30 19:38:27.157295 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.155540    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8399c42-686d-4164-b96d-e61f6a99c8bd-istio-data\") pod \"e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") " pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"
Oct 30 19:38:27.157295 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.155612    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8399c42-686d-4164-b96d-e61f6a99c8bd-workload-certs\") pod \"e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") " pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"
Oct 30 19:38:27.157295 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.155653    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8399c42-686d-4164-b96d-e61f6a99c8bd-istio-podinfo\") pod \"e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") " pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"
Oct 30 19:38:27.157295 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.155750    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8399c42-686d-4164-b96d-e61f6a99c8bd-workload-socket\") pod \"e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") " pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"
Oct 30 19:38:27.157295 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.156004    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8399c42-686d-4164-b96d-e61f6a99c8bd-istio-token\") pod \"e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") " pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"
Oct 30 19:38:27.157295 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.156087    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8399c42-686d-4164-b96d-e61f6a99c8bd-istio-envoy\") pod \"e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") " pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"
Oct 30 19:38:27.157295 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.156185    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3c0d05d-67f1-4e07-82d0-516ff973b2f2-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:27.157295 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.156203    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7659a68f-a908-46c0-a61c-74f827b84339-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:27.157295 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.156639    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8399c42-686d-4164-b96d-e61f6a99c8bd-istiod-ca-cert\") pod \"e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") " pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"
Oct 30 19:38:27.157295 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.156804    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8399c42-686d-4164-b96d-e61f6a99c8bd-credential-socket\") pod \"e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") " pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"
Oct 30 19:38:27.157295 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.157139    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8399c42-686d-4164-b96d-e61f6a99c8bd-istio-data\") pod \"e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") " pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"
Oct 30 19:38:27.158174 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.158124    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8399c42-686d-4164-b96d-e61f6a99c8bd-workload-socket\") pod \"e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") " pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"
Oct 30 19:38:27.161143 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.161114    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8399c42-686d-4164-b96d-e61f6a99c8bd-workload-certs\") pod \"e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") " pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"
Oct 30 19:38:27.163608 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.162923    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8399c42-686d-4164-b96d-e61f6a99c8bd-istio-envoy\") pod \"e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") " pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"
Oct 30 19:38:27.166175 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.165717    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3c0d05d-67f1-4e07-82d0-516ff973b2f2-kube-api-access-h6flk" pod "a3c0d05d-67f1-4e07-82d0-516ff973b2f2" (UID: "a3c0d05d-67f1-4e07-82d0-516ff973b2f2"). InnerVolumeSpecName "kube-api-access-h6flk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:27.166175 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.166133    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8399c42-686d-4164-b96d-e61f6a99c8bd-istio-podinfo\") pod \"e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") " pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"
Oct 30 19:38:27.168220 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.168193    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7659a68f-a908-46c0-a61c-74f827b84339-kube-api-access-cdbr5" pod "7659a68f-a908-46c0-a61c-74f827b84339" (UID: "7659a68f-a908-46c0-a61c-74f827b84339"). InnerVolumeSpecName "kube-api-access-cdbr5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:27.179849 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.179644    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8ldg\" (UniqueName: \"kubernetes.io/projected/c8399c42-686d-4164-b96d-e61f6a99c8bd-kube-api-access-n8ldg\") pod \"e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") " pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"
Oct 30 19:38:27.179849 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.179802    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8399c42-686d-4164-b96d-e61f6a99c8bd-istio-token\") pod \"e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") " pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"
Oct 30 19:38:27.209610 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.209180    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-df776e5c-57c7bd5f5f-rvsvz"
Oct 30 19:38:27.257680 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.257637    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f2c3dee9-2c52-44a2-a64e-aa661cce2452-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2c3dee9-2c52-44a2-a64e-aa661cce2452-plugins-config-volume\") pod \"f2c3dee9-2c52-44a2-a64e-aa661cce2452\" (UID: \"f2c3dee9-2c52-44a2-a64e-aa661cce2452\") "
Oct 30 19:38:27.257994 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.257969    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f2c3dee9-2c52-44a2-a64e-aa661cce2452-kube-api-access-54wd5\" (UniqueName: \"kubernetes.io/projected/f2c3dee9-2c52-44a2-a64e-aa661cce2452-kube-api-access-54wd5\") pod \"f2c3dee9-2c52-44a2-a64e-aa661cce2452\" (UID: \"f2c3dee9-2c52-44a2-a64e-aa661cce2452\") "
Oct 30 19:38:27.258717 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.258692    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6flk\" (UniqueName: \"kubernetes.io/projected/a3c0d05d-67f1-4e07-82d0-516ff973b2f2-kube-api-access-h6flk\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:27.258811 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.258719    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdbr5\" (UniqueName: \"kubernetes.io/projected/7659a68f-a908-46c0-a61c-74f827b84339-kube-api-access-cdbr5\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:27.259703 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.259677    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2c3dee9-2c52-44a2-a64e-aa661cce2452-plugins-config-volume" pod "f2c3dee9-2c52-44a2-a64e-aa661cce2452" (UID: "f2c3dee9-2c52-44a2-a64e-aa661cce2452"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:38:27.261669 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.261640    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2c3dee9-2c52-44a2-a64e-aa661cce2452-kube-api-access-54wd5" pod "f2c3dee9-2c52-44a2-a64e-aa661cce2452" (UID: "f2c3dee9-2c52-44a2-a64e-aa661cce2452"). InnerVolumeSpecName "kube-api-access-54wd5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:27.360730 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.360523    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2c3dee9-2c52-44a2-a64e-aa661cce2452-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:27.360730 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.360546    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54wd5\" (UniqueName: \"kubernetes.io/projected/f2c3dee9-2c52-44a2-a64e-aa661cce2452-kube-api-access-54wd5\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:27.372832 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.372799    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"
Oct 30 19:38:27.389753 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.389725    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-df776e5c-6b7db86b98-xbxfg"
Oct 30 19:38:27.463941 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.462364    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/207b8150-f938-4bce-a4d6-473fdf54212d-kube-api-access-8cggx\" (UniqueName: \"kubernetes.io/projected/207b8150-f938-4bce-a4d6-473fdf54212d-kube-api-access-8cggx\") pod \"207b8150-f938-4bce-a4d6-473fdf54212d\" (UID: \"207b8150-f938-4bce-a4d6-473fdf54212d\") "
Oct 30 19:38:27.463941 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.462437    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/207b8150-f938-4bce-a4d6-473fdf54212d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/207b8150-f938-4bce-a4d6-473fdf54212d-plugins-config-volume\") pod \"207b8150-f938-4bce-a4d6-473fdf54212d\" (UID: \"207b8150-f938-4bce-a4d6-473fdf54212d\") "
Oct 30 19:38:27.509891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.505700    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/207b8150-f938-4bce-a4d6-473fdf54212d-plugins-config-volume" pod "207b8150-f938-4bce-a4d6-473fdf54212d" (UID: "207b8150-f938-4bce-a4d6-473fdf54212d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:38:27.611888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.573725    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/207b8150-f938-4bce-a4d6-473fdf54212d-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:27.630106 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.623356    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:38:27.680408 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.676014    5907 patch_prober.go:29] interesting pod/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 30 19:38:27.680408 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.676089    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh" podUID="ce9284b5-c1c6-4390-a5aa-410c7538e9eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 30 19:38:27.765230 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.765113    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/207b8150-f938-4bce-a4d6-473fdf54212d-kube-api-access-8cggx" pod "207b8150-f938-4bce-a4d6-473fdf54212d" (UID: "207b8150-f938-4bce-a4d6-473fdf54212d"). InnerVolumeSpecName "kube-api-access-8cggx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:27.848037 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.847981    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8cggx\" (UniqueName: \"kubernetes.io/projected/207b8150-f938-4bce-a4d6-473fdf54212d-kube-api-access-8cggx\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:27.988874 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.988830    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5016f41a-bd5df848c-mvxr6"
Oct 30 19:38:27.989327 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.988980    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5016f41a-bd5df848c-mvxr6" event={"ID":"7659a68f-a908-46c0-a61c-74f827b84339","Type":"ContainerDied","Data":"670920b46a809adcefc95da9673fa7c5f1d49db3387342f82695ab2c90ec2989"}
Oct 30 19:38:27.991642 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.991587    5907 generic.go:356] "Generic (PLEG): container finished" podID="207b8150-f938-4bce-a4d6-473fdf54212d" containerID="209f228f2e91ea65e264665577cd1fa7e92de54f272c298e6d1f39b9bfbed352" exitCode=0
Oct 30 19:38:27.991806 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.991716    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-df776e5c-6b7db86b98-xbxfg" event={"ID":"207b8150-f938-4bce-a4d6-473fdf54212d","Type":"ContainerDied","Data":"209f228f2e91ea65e264665577cd1fa7e92de54f272c298e6d1f39b9bfbed352"}
Oct 30 19:38:27.991806 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.991782    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-df776e5c-6b7db86b98-xbxfg" event={"ID":"207b8150-f938-4bce-a4d6-473fdf54212d","Type":"ContainerDied","Data":"fe71d22aa8321d4bbb39de3e0ecad8633640eca9d58b85a55f822c3f63983925"}
Oct 30 19:38:27.991946 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.991822    5907 scope.go:122] "RemoveContainer" containerID="209f228f2e91ea65e264665577cd1fa7e92de54f272c298e6d1f39b9bfbed352"
Oct 30 19:38:27.994911 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.994001    5907 generic.go:356] "Generic (PLEG): container finished" podID="f2c3dee9-2c52-44a2-a64e-aa661cce2452" containerID="0c14c210a21d68514911ce66e7caa7132f66fbf30f76c2e7c62834fd9e369e9d" exitCode=0
Oct 30 19:38:27.994911 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.994098    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-df776e5c-57c7bd5f5f-rvsvz" event={"ID":"f2c3dee9-2c52-44a2-a64e-aa661cce2452","Type":"ContainerDied","Data":"0c14c210a21d68514911ce66e7caa7132f66fbf30f76c2e7c62834fd9e369e9d"}
Oct 30 19:38:27.994911 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.994126    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-df776e5c-57c7bd5f5f-rvsvz" event={"ID":"f2c3dee9-2c52-44a2-a64e-aa661cce2452","Type":"ContainerDied","Data":"0dcbb4362752cca94225e7e16930374daaaad28a531a2d69b0a1bbf347dec41f"}
Oct 30 19:38:27.994911 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.994186    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-df776e5c-57c7bd5f5f-rvsvz"
Oct 30 19:38:27.995896 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.995622    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-df776e5c-6b7db86b98-xbxfg"
Oct 30 19:38:27.997930 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.997401    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-5016f41a-bff455988-57j6h"
Oct 30 19:38:27.997930 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:27.997581    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-5016f41a-bff455988-57j6h" event={"ID":"a3c0d05d-67f1-4e07-82d0-516ff973b2f2","Type":"ContainerDied","Data":"66185f3cd364f3e6ecf8dd7790138e204874c0a6d4934dbe8f0e76a2d80d6def"}
Oct 30 19:38:28.194195 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:28.194163    5907 scope.go:122] "RemoveContainer" containerID="209f228f2e91ea65e264665577cd1fa7e92de54f272c298e6d1f39b9bfbed352"
Oct 30 19:38:28.194943 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:38:28.194914    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"209f228f2e91ea65e264665577cd1fa7e92de54f272c298e6d1f39b9bfbed352\": container with ID starting with 209f228f2e91ea65e264665577cd1fa7e92de54f272c298e6d1f39b9bfbed352 not found: ID does not exist" containerID="209f228f2e91ea65e264665577cd1fa7e92de54f272c298e6d1f39b9bfbed352"
Oct 30 19:38:28.195149 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:28.195098    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"209f228f2e91ea65e264665577cd1fa7e92de54f272c298e6d1f39b9bfbed352"} err="failed to get container status \"209f228f2e91ea65e264665577cd1fa7e92de54f272c298e6d1f39b9bfbed352\": rpc error: code = NotFound desc = could not find container \"209f228f2e91ea65e264665577cd1fa7e92de54f272c298e6d1f39b9bfbed352\": container with ID starting with 209f228f2e91ea65e264665577cd1fa7e92de54f272c298e6d1f39b9bfbed352 not found: ID does not exist"
Oct 30 19:38:28.195268 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:28.195256    5907 scope.go:122] "RemoveContainer" containerID="0c14c210a21d68514911ce66e7caa7132f66fbf30f76c2e7c62834fd9e369e9d"
Oct 30 19:38:28.270301 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:28.270178    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:38:28.270301 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:28.270260    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:38:28.270490 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:28.270304    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:38:28.272431 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:28.272408    5907 scope.go:122] "RemoveContainer" containerID="0c14c210a21d68514911ce66e7caa7132f66fbf30f76c2e7c62834fd9e369e9d"
Oct 30 19:38:28.273041 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:38:28.272947    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c14c210a21d68514911ce66e7caa7132f66fbf30f76c2e7c62834fd9e369e9d\": container with ID starting with 0c14c210a21d68514911ce66e7caa7132f66fbf30f76c2e7c62834fd9e369e9d not found: ID does not exist" containerID="0c14c210a21d68514911ce66e7caa7132f66fbf30f76c2e7c62834fd9e369e9d"
Oct 30 19:38:28.273041 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:28.272981    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c14c210a21d68514911ce66e7caa7132f66fbf30f76c2e7c62834fd9e369e9d"} err="failed to get container status \"0c14c210a21d68514911ce66e7caa7132f66fbf30f76c2e7c62834fd9e369e9d\": rpc error: code = NotFound desc = could not find container \"0c14c210a21d68514911ce66e7caa7132f66fbf30f76c2e7c62834fd9e369e9d\": container with ID starting with 0c14c210a21d68514911ce66e7caa7132f66fbf30f76c2e7c62834fd9e369e9d not found: ID does not exist"
Oct 30 19:38:28.279119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:28.277442    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5016f41a-d492t"]
Oct 30 19:38:28.303456 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:28.303409    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a071c805-8252-48da-ac16-d4f73a1b15cc" path="/var/lib/kubelet/pods/a071c805-8252-48da-ac16-d4f73a1b15cc/volumes"
Oct 30 19:38:28.304432 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:28.304410    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5016f41a-d492t"]
Oct 30 19:38:28.304575 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:28.304564    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"]
Oct 30 19:38:28.356760 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:28.356667    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"]
Oct 30 19:38:28.368801 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:38:28.368748    5907 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod207b8150_f938_4bce_a4d6_473fdf54212d.slice/crio-fe71d22aa8321d4bbb39de3e0ecad8633640eca9d58b85a55f822c3f63983925\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7659a68f_a908_46c0_a61c_74f827b84339.slice/crio-670920b46a809adcefc95da9673fa7c5f1d49db3387342f82695ab2c90ec2989\": RecentStats: unable to find data in memory cache]"
Oct 30 19:38:28.593976 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:28.593925    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-5016f41a-bff455988-57j6h"]
Oct 30 19:38:28.617902 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:28.612793    5907 patch_prober.go:29] interesting pod/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 30 19:38:28.617902 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:28.612852    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh" podUID="ce9284b5-c1c6-4390-a5aa-410c7538e9eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 30 19:38:29.062395 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:29.061733    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-5016f41a-bff455988-57j6h"]
Oct 30 19:38:29.069568 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:29.069539    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-5016f41a-bd5df848c-mvxr6"]
Oct 30 19:38:29.075614 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:29.075575    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh" podUID="ce9284b5-c1c6-4390-a5aa-410c7538e9eb" containerName="istio-proxy" containerID="cri-o://725d58d2a861352576472103d26ce61f093b5ab05c768b1d41bf8aacd7311c9c" gracePeriod=30
Oct 30 19:38:29.077189 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:29.076616    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r" event={"ID":"c8399c42-686d-4164-b96d-e61f6a99c8bd","Type":"ContainerStarted","Data":"befb5c826eb7e3be2ed79a72442967a477fa7247a3b0c7cf1b2da0c628e98c36"}
Oct 30 19:38:29.077189 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:29.076668    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r" event={"ID":"c8399c42-686d-4164-b96d-e61f6a99c8bd","Type":"ContainerStarted","Data":"43366102b6b3d6e34fc06b2d18ba459f675122016df9cc39a01f308546eea025"}
Oct 30 19:38:29.085788 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:29.080993    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-5016f41a-bd5df848c-mvxr6"]
Oct 30 19:38:29.088200 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:29.088174    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-df776e5c-57c7bd5f5f-rvsvz"]
Oct 30 19:38:29.096250 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:29.096184    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-df776e5c-57c7bd5f5f-rvsvz"]
Oct 30 19:38:29.107630 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:29.107339    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-df776e5c-6b7db86b98-xbxfg"]
Oct 30 19:38:29.163681 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:29.163650    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-df776e5c-6b7db86b98-xbxfg"]
Oct 30 19:38:29.176542 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:29.176514    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-379c7cd1-zfmk8"]
Oct 30 19:38:29.180921 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:29.180896    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-379c7cd1-zfmk8"]
Oct 30 19:38:29.182718 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:29.182676    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r" podStartSLOduration=4.182663991 podStartE2EDuration="4.182663991s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:38:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:38:29.129795065 +0000 UTC m=+1020.972727048" watchObservedRunningTime="2026-10-30 19:38:29.182663991 +0000 UTC m=+1021.025595995"
Oct 30 19:38:29.380883 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:29.377110    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"
Oct 30 19:38:29.380883 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:29.377280    5907 patch_prober.go:29] interesting pod/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused" start-of-body=
Oct 30 19:38:29.380883 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:29.377324    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r" podUID="c8399c42-686d-4164-b96d-e61f6a99c8bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused"
Oct 30 19:38:30.291070 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:30.291034    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="207b8150-f938-4bce-a4d6-473fdf54212d" path="/var/lib/kubelet/pods/207b8150-f938-4bce-a4d6-473fdf54212d/volumes"
Oct 30 19:38:30.291706 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:30.291685    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7659a68f-a908-46c0-a61c-74f827b84339" path="/var/lib/kubelet/pods/7659a68f-a908-46c0-a61c-74f827b84339/volumes"
Oct 30 19:38:30.292169 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:30.292149    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a3c0d05d-67f1-4e07-82d0-516ff973b2f2" path="/var/lib/kubelet/pods/a3c0d05d-67f1-4e07-82d0-516ff973b2f2/volumes"
Oct 30 19:38:30.292938 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:30.292913    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c5d874c6-d340-4b99-999a-dd4a8922f577" path="/var/lib/kubelet/pods/c5d874c6-d340-4b99-999a-dd4a8922f577/volumes"
Oct 30 19:38:30.293420 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:30.293399    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ecf613ff-7fd9-462f-af0b-95256bf973f9" path="/var/lib/kubelet/pods/ecf613ff-7fd9-462f-af0b-95256bf973f9/volumes"
Oct 30 19:38:30.293892 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:30.293853    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f2c3dee9-2c52-44a2-a64e-aa661cce2452" path="/var/lib/kubelet/pods/f2c3dee9-2c52-44a2-a64e-aa661cce2452/volumes"
Oct 30 19:38:30.378251 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:30.378221    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"
Oct 30 19:38:31.001887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:30.996039    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:38:31.061540 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:31.061504    5907 patch_prober.go:29] interesting pod/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.135:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:38:31.061812 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:31.061783    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz" podUID="a28e0e27-b1cb-4d23-a869-a291ccfd30ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.135:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:38:31.353014 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:31.352924    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-fa5a7c92-6vf75"]
Oct 30 19:38:31.373004 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:31.372971    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"
Oct 30 19:38:31.373297 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:31.373282    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"
Oct 30 19:38:31.483903 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:31.471196    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-fa5a7c92-6vf75"]
Oct 30 19:38:31.983121 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:31.983034    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-df776e5c-2nbtv"]
Oct 30 19:38:32.006736 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:32.002588    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-df776e5c-2nbtv"]
Oct 30 19:38:32.061132 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:32.058482    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-68a9d230-zz95v"]
Oct 30 19:38:32.066138 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:32.066104    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-68a9d230-zz95v"]
Oct 30 19:38:32.692390 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:32.692345    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ac1c865a-3a93-43d9-a764-902a86c9aa1b" path="/var/lib/kubelet/pods/ac1c865a-3a93-43d9-a764-902a86c9aa1b/volumes"
Oct 30 19:38:32.694126 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:32.694098    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cd697abb-3adc-4a4d-b30b-44fb8dea4b7f" path="/var/lib/kubelet/pods/cd697abb-3adc-4a4d-b30b-44fb8dea4b7f/volumes"
Oct 30 19:38:32.695232 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:32.695209    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a" path="/var/lib/kubelet/pods/cf4fdff0-0f8b-41b7-86ab-5df6fe066f4a/volumes"
Oct 30 19:38:32.748886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:32.744956    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-46bd1516-hxjfq"]
Oct 30 19:38:32.748886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:32.747114    5907 cpu_manager.go:414] "containerMap: removing container" podUID="207b8150-f938-4bce-a4d6-473fdf54212d" containerName="payload-processing"
Oct 30 19:38:32.748886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:32.747139    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="207b8150-f938-4bce-a4d6-473fdf54212d" containerName="payload-processing"
Oct 30 19:38:32.748886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:32.747215    5907 cpu_manager.go:414] "containerMap: removing container" podUID="f2c3dee9-2c52-44a2-a64e-aa661cce2452" containerName="payload-pre-processing"
Oct 30 19:38:32.748886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:32.747224    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f2c3dee9-2c52-44a2-a64e-aa661cce2452" containerName="payload-pre-processing"
Oct 30 19:38:32.748886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:32.747642    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="207b8150-f938-4bce-a4d6-473fdf54212d" containerName="payload-processing"
Oct 30 19:38:32.748886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:32.747672    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="f2c3dee9-2c52-44a2-a64e-aa661cce2452" containerName="payload-pre-processing"
Oct 30 19:38:32.780541 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:32.778168    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-46bd1516-hxjfq"]
Oct 30 19:38:32.780541 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:32.778293    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-46bd1516-hxjfq"
Oct 30 19:38:32.797591 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:32.797433    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:38:32.797591 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:32.797562    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:38:32.904782 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:32.904739    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djqdw\" (UniqueName: \"kubernetes.io/projected/956f186f-15d0-4f15-806e-4ad518542d35-kube-api-access-djqdw\") pod \"maas-api-revoke-keys-e2e-ait-46bd1516-hxjfq\" (UID: \"956f186f-15d0-4f15-806e-4ad518542d35\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-46bd1516-hxjfq"
Oct 30 19:38:32.904999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:32.904942    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/956f186f-15d0-4f15-806e-4ad518542d35-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-46bd1516-hxjfq\" (UID: \"956f186f-15d0-4f15-806e-4ad518542d35\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-46bd1516-hxjfq"
Oct 30 19:38:32.905218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:32.905190    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/956f186f-15d0-4f15-806e-4ad518542d35-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-46bd1516-hxjfq\" (UID: \"956f186f-15d0-4f15-806e-4ad518542d35\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-46bd1516-hxjfq"
Oct 30 19:38:33.007451 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:33.007352    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/956f186f-15d0-4f15-806e-4ad518542d35-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-46bd1516-hxjfq\" (UID: \"956f186f-15d0-4f15-806e-4ad518542d35\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-46bd1516-hxjfq"
Oct 30 19:38:33.007607 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:33.007591    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djqdw\" (UniqueName: \"kubernetes.io/projected/956f186f-15d0-4f15-806e-4ad518542d35-kube-api-access-djqdw\") pod \"maas-api-revoke-keys-e2e-ait-46bd1516-hxjfq\" (UID: \"956f186f-15d0-4f15-806e-4ad518542d35\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-46bd1516-hxjfq"
Oct 30 19:38:33.007703 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:33.007689    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/956f186f-15d0-4f15-806e-4ad518542d35-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-46bd1516-hxjfq\" (UID: \"956f186f-15d0-4f15-806e-4ad518542d35\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-46bd1516-hxjfq"
Oct 30 19:38:33.008091 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:33.008066    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/956f186f-15d0-4f15-806e-4ad518542d35-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-46bd1516-hxjfq\" (UID: \"956f186f-15d0-4f15-806e-4ad518542d35\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-46bd1516-hxjfq"
Oct 30 19:38:33.035918 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:33.030611    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/956f186f-15d0-4f15-806e-4ad518542d35-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-46bd1516-hxjfq\" (UID: \"956f186f-15d0-4f15-806e-4ad518542d35\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-46bd1516-hxjfq"
Oct 30 19:38:33.038838 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:33.037984    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djqdw\" (UniqueName: \"kubernetes.io/projected/956f186f-15d0-4f15-806e-4ad518542d35-kube-api-access-djqdw\") pod \"maas-api-revoke-keys-e2e-ait-46bd1516-hxjfq\" (UID: \"956f186f-15d0-4f15-806e-4ad518542d35\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-46bd1516-hxjfq"
Oct 30 19:38:33.051104 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:33.051064    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6323c400-jpdh2"]
Oct 30 19:38:33.058649 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:33.058608    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6323c400-jpdh2"]
Oct 30 19:38:33.103849 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:33.103810    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-46bd1516-hxjfq"
Oct 30 19:38:33.455929 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:38:33.455900    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod956f186f_15d0_4f15_806e_4ad518542d35.slice/crio-77296f885e8915aa7f1a21e85b120f0998c1b3816edbb77718a54693d812a109 WatchSource:0}: Error finding container 77296f885e8915aa7f1a21e85b120f0998c1b3816edbb77718a54693d812a109: Status 404 returned error can't find the container with id 77296f885e8915aa7f1a21e85b120f0998c1b3816edbb77718a54693d812a109
Oct 30 19:38:33.460727 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:33.460701    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-46bd1516-hxjfq"]
Oct 30 19:38:33.661622 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:33.661588    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-46bd1516-hxjfq" event={"ID":"956f186f-15d0-4f15-806e-4ad518542d35","Type":"ContainerStarted","Data":"77296f885e8915aa7f1a21e85b120f0998c1b3816edbb77718a54693d812a109"}
Oct 30 19:38:33.935934 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:33.935898    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b9c43c35-66d656d8f9-7qxjn"]
Oct 30 19:38:33.945567 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:33.944760    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b9c43c35-66d656d8f9-7qxjn"
Oct 30 19:38:33.956786 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:33.956762    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b9c43c35-66d656d8f9-7qxjn"]
Oct 30 19:38:33.960544 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:33.960514    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-b9c43c35-dockercfg-c5rxp\""
Oct 30 19:38:33.960682 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:33.960617    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-b9c43c35\""
Oct 30 19:38:34.103344 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:34.070968    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de43dd5e-6749-475c-b56e-c2fdba78aba6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b9c43c35-66d656d8f9-7qxjn\" (UID: \"de43dd5e-6749-475c-b56e-c2fdba78aba6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b9c43c35-66d656d8f9-7qxjn"
Oct 30 19:38:34.103344 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:34.071029    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhz69\" (UniqueName: \"kubernetes.io/projected/de43dd5e-6749-475c-b56e-c2fdba78aba6-kube-api-access-jhz69\") pod \"payload-pre-processing-e2e-ait-b9c43c35-66d656d8f9-7qxjn\" (UID: \"de43dd5e-6749-475c-b56e-c2fdba78aba6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b9c43c35-66d656d8f9-7qxjn"
Oct 30 19:38:34.400152 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:34.174341    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de43dd5e-6749-475c-b56e-c2fdba78aba6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b9c43c35-66d656d8f9-7qxjn\" (UID: \"de43dd5e-6749-475c-b56e-c2fdba78aba6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b9c43c35-66d656d8f9-7qxjn"
Oct 30 19:38:34.400152 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:34.174416    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhz69\" (UniqueName: \"kubernetes.io/projected/de43dd5e-6749-475c-b56e-c2fdba78aba6-kube-api-access-jhz69\") pod \"payload-pre-processing-e2e-ait-b9c43c35-66d656d8f9-7qxjn\" (UID: \"de43dd5e-6749-475c-b56e-c2fdba78aba6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b9c43c35-66d656d8f9-7qxjn"
Oct 30 19:38:34.400152 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:34.175757    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de43dd5e-6749-475c-b56e-c2fdba78aba6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b9c43c35-66d656d8f9-7qxjn\" (UID: \"de43dd5e-6749-475c-b56e-c2fdba78aba6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b9c43c35-66d656d8f9-7qxjn"
Oct 30 19:38:34.400152 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:34.316079    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b9c43c35-7dc4b6fdf8-v4zvp"]
Oct 30 19:38:34.400152 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:34.372432    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b9c43c35-7dc4b6fdf8-v4zvp"
Oct 30 19:38:34.427129 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:34.422294    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vd2vq\" (UniqueName: \"kubernetes.io/projected/f39a79fa-188d-451c-af60-7a69d653d136-kube-api-access-vd2vq\") pod \"payload-processing-e2e-ait-b9c43c35-7dc4b6fdf8-v4zvp\" (UID: \"f39a79fa-188d-451c-af60-7a69d653d136\") " pod="openshift-ingress/payload-processing-e2e-ait-b9c43c35-7dc4b6fdf8-v4zvp"
Oct 30 19:38:34.427129 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:34.422452    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f39a79fa-188d-451c-af60-7a69d653d136-plugins-config-volume\") pod \"payload-processing-e2e-ait-b9c43c35-7dc4b6fdf8-v4zvp\" (UID: \"f39a79fa-188d-451c-af60-7a69d653d136\") " pod="openshift-ingress/payload-processing-e2e-ait-b9c43c35-7dc4b6fdf8-v4zvp"
Oct 30 19:38:34.451895 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:34.442182    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5b5d8900-05d1-4438-bc0f-2de9c7890ae7" path="/var/lib/kubelet/pods/5b5d8900-05d1-4438-bc0f-2de9c7890ae7/volumes"
Oct 30 19:38:34.451895 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:34.442674    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b9c43c35-7dc4b6fdf8-v4zvp"]
Oct 30 19:38:34.473723 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:34.468249    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhz69\" (UniqueName: \"kubernetes.io/projected/de43dd5e-6749-475c-b56e-c2fdba78aba6-kube-api-access-jhz69\") pod \"payload-pre-processing-e2e-ait-b9c43c35-66d656d8f9-7qxjn\" (UID: \"de43dd5e-6749-475c-b56e-c2fdba78aba6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b9c43c35-66d656d8f9-7qxjn"
Oct 30 19:38:34.524769 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:34.524727    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vd2vq\" (UniqueName: \"kubernetes.io/projected/f39a79fa-188d-451c-af60-7a69d653d136-kube-api-access-vd2vq\") pod \"payload-processing-e2e-ait-b9c43c35-7dc4b6fdf8-v4zvp\" (UID: \"f39a79fa-188d-451c-af60-7a69d653d136\") " pod="openshift-ingress/payload-processing-e2e-ait-b9c43c35-7dc4b6fdf8-v4zvp"
Oct 30 19:38:34.525006 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:34.524982    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f39a79fa-188d-451c-af60-7a69d653d136-plugins-config-volume\") pod \"payload-processing-e2e-ait-b9c43c35-7dc4b6fdf8-v4zvp\" (UID: \"f39a79fa-188d-451c-af60-7a69d653d136\") " pod="openshift-ingress/payload-processing-e2e-ait-b9c43c35-7dc4b6fdf8-v4zvp"
Oct 30 19:38:34.526503 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:34.526473    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f39a79fa-188d-451c-af60-7a69d653d136-plugins-config-volume\") pod \"payload-processing-e2e-ait-b9c43c35-7dc4b6fdf8-v4zvp\" (UID: \"f39a79fa-188d-451c-af60-7a69d653d136\") " pod="openshift-ingress/payload-processing-e2e-ait-b9c43c35-7dc4b6fdf8-v4zvp"
Oct 30 19:38:34.547353 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:34.547315    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vd2vq\" (UniqueName: \"kubernetes.io/projected/f39a79fa-188d-451c-af60-7a69d653d136-kube-api-access-vd2vq\") pod \"payload-processing-e2e-ait-b9c43c35-7dc4b6fdf8-v4zvp\" (UID: \"f39a79fa-188d-451c-af60-7a69d653d136\") " pod="openshift-ingress/payload-processing-e2e-ait-b9c43c35-7dc4b6fdf8-v4zvp"
Oct 30 19:38:34.686715 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:34.686427    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b9c43c35-66d656d8f9-7qxjn"
Oct 30 19:38:34.777935 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:34.777448    5907 generic.go:356] "Generic (PLEG): container finished" podID="956f186f-15d0-4f15-806e-4ad518542d35" containerID="78b5f480af5e2be25256646ae63185435b206608f477a40828980aa6007f2c74" exitCode=0
Oct 30 19:38:34.777935 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:34.777803    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-46bd1516-hxjfq" event={"ID":"956f186f-15d0-4f15-806e-4ad518542d35","Type":"ContainerDied","Data":"78b5f480af5e2be25256646ae63185435b206608f477a40828980aa6007f2c74"}
Oct 30 19:38:34.831190 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:34.831061    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b9c43c35-7dc4b6fdf8-v4zvp"
Oct 30 19:38:35.090755 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:38:35.090714    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde43dd5e_6749_475c_b56e_c2fdba78aba6.slice/crio-69f658fc17cd5d5a443d533cb39701be7388e94507780b08a31b1cf985960936 WatchSource:0}: Error finding container 69f658fc17cd5d5a443d533cb39701be7388e94507780b08a31b1cf985960936: Status 404 returned error can't find the container with id 69f658fc17cd5d5a443d533cb39701be7388e94507780b08a31b1cf985960936
Oct 30 19:38:35.093850 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:35.092620    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b9c43c35-66d656d8f9-7qxjn"]
Oct 30 19:38:35.172888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:35.172827    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b9c43c35-7dc4b6fdf8-v4zvp"]
Oct 30 19:38:35.174179 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:38:35.174153    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf39a79fa_188d_451c_af60_7a69d653d136.slice/crio-d816be2bf7f83eaa95f3e8b76c012a99e2eaf719045885c02793353bb9699bcb WatchSource:0}: Error finding container d816be2bf7f83eaa95f3e8b76c012a99e2eaf719045885c02793353bb9699bcb: Status 404 returned error can't find the container with id d816be2bf7f83eaa95f3e8b76c012a99e2eaf719045885c02793353bb9699bcb
Oct 30 19:38:35.793006 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:35.792962    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b9c43c35-7dc4b6fdf8-v4zvp" event={"ID":"f39a79fa-188d-451c-af60-7a69d653d136","Type":"ContainerStarted","Data":"a6ca102d4b7091c9ea905acf6da685e1b1c0e11d31efe9efaa7d4e1735f7f4ba"}
Oct 30 19:38:35.793217 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:35.793071    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b9c43c35-7dc4b6fdf8-v4zvp" event={"ID":"f39a79fa-188d-451c-af60-7a69d653d136","Type":"ContainerStarted","Data":"d816be2bf7f83eaa95f3e8b76c012a99e2eaf719045885c02793353bb9699bcb"}
Oct 30 19:38:35.793217 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:35.793102    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-b9c43c35-7dc4b6fdf8-v4zvp"
Oct 30 19:38:35.795227 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:35.795195    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b9c43c35-66d656d8f9-7qxjn" event={"ID":"de43dd5e-6749-475c-b56e-c2fdba78aba6","Type":"ContainerStarted","Data":"c2d3fa15829fca941d95f266c0df27a78c23fe081af032809bafad180ba0a832"}
Oct 30 19:38:35.795342 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:35.795253    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b9c43c35-66d656d8f9-7qxjn" event={"ID":"de43dd5e-6749-475c-b56e-c2fdba78aba6","Type":"ContainerStarted","Data":"69f658fc17cd5d5a443d533cb39701be7388e94507780b08a31b1cf985960936"}
Oct 30 19:38:35.795441 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:35.795417    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-b9c43c35-66d656d8f9-7qxjn"
Oct 30 19:38:35.835513 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:35.835198    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-b9c43c35-7dc4b6fdf8-v4zvp" podStartSLOduration=2.835182349 podStartE2EDuration="2.835182349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:38:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:38:35.834493612 +0000 UTC m=+1027.677425563" watchObservedRunningTime="2026-10-30 19:38:35.835182349 +0000 UTC m=+1027.678114288"
Oct 30 19:38:35.880131 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:35.879675    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-b9c43c35-66d656d8f9-7qxjn" podStartSLOduration=2.879661814 podStartE2EDuration="2.879661814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:38:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:38:35.877662403 +0000 UTC m=+1027.720594345" watchObservedRunningTime="2026-10-30 19:38:35.879661814 +0000 UTC m=+1027.722593753"
Oct 30 19:38:35.948985 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:35.948960    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-46bd1516-hxjfq"
Oct 30 19:38:36.117664 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.080661    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/956f186f-15d0-4f15-806e-4ad518542d35-kube-api-access-djqdw\" (UniqueName: \"kubernetes.io/projected/956f186f-15d0-4f15-806e-4ad518542d35-kube-api-access-djqdw\") pod \"956f186f-15d0-4f15-806e-4ad518542d35\" (UID: \"956f186f-15d0-4f15-806e-4ad518542d35\") "
Oct 30 19:38:36.117664 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.080778    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/956f186f-15d0-4f15-806e-4ad518542d35-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/956f186f-15d0-4f15-806e-4ad518542d35-maas-api-xks-ca\") pod \"956f186f-15d0-4f15-806e-4ad518542d35\" (UID: \"956f186f-15d0-4f15-806e-4ad518542d35\") "
Oct 30 19:38:36.117664 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.080848    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/956f186f-15d0-4f15-806e-4ad518542d35-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/956f186f-15d0-4f15-806e-4ad518542d35-maas-api-openshift-ca\") pod \"956f186f-15d0-4f15-806e-4ad518542d35\" (UID: \"956f186f-15d0-4f15-806e-4ad518542d35\") "
Oct 30 19:38:36.127549 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.127165    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/956f186f-15d0-4f15-806e-4ad518542d35-maas-api-openshift-ca" pod "956f186f-15d0-4f15-806e-4ad518542d35" (UID: "956f186f-15d0-4f15-806e-4ad518542d35"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:38:36.162985 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.153957    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/956f186f-15d0-4f15-806e-4ad518542d35-kube-api-access-djqdw" pod "956f186f-15d0-4f15-806e-4ad518542d35" (UID: "956f186f-15d0-4f15-806e-4ad518542d35"). InnerVolumeSpecName "kube-api-access-djqdw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:36.170643 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.163552    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/956f186f-15d0-4f15-806e-4ad518542d35-maas-api-xks-ca" pod "956f186f-15d0-4f15-806e-4ad518542d35" (UID: "956f186f-15d0-4f15-806e-4ad518542d35"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:38:36.195273 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.194984    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/956f186f-15d0-4f15-806e-4ad518542d35-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:36.195273 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.195021    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/956f186f-15d0-4f15-806e-4ad518542d35-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:36.195273 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.195036    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djqdw\" (UniqueName: \"kubernetes.io/projected/956f186f-15d0-4f15-806e-4ad518542d35-kube-api-access-djqdw\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:36.402089 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.402063    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"
Oct 30 19:38:36.499240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.499207    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-workload-certs\") pod \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") "
Oct 30 19:38:36.499409 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.499278    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-istio-token\" (UniqueName: \"kubernetes.io/projected/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-istio-token\") pod \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") "
Oct 30 19:38:36.499409 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.499307    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-istio-envoy\") pod \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") "
Oct 30 19:38:36.499409 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.499344    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-istio-data\") pod \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") "
Oct 30 19:38:36.499548 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.499504    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-workload-certs" pod "a28e0e27-b1cb-4d23-a869-a291ccfd30ae" (UID: "a28e0e27-b1cb-4d23-a869-a291ccfd30ae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:36.499548 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.499517    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-kube-api-access-8nbgt\" (UniqueName: \"kubernetes.io/projected/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-kube-api-access-8nbgt\") pod \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") "
Oct 30 19:38:36.499652 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.499630    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-workload-socket\") pod \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") "
Oct 30 19:38:36.499719 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.499663    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-istiod-ca-cert\") pod \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") "
Oct 30 19:38:36.499719 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.499708    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-istio-data" pod "a28e0e27-b1cb-4d23-a869-a291ccfd30ae" (UID: "a28e0e27-b1cb-4d23-a869-a291ccfd30ae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:36.499833 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.499744    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-credential-socket\") pod \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") "
Oct 30 19:38:36.499833 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.499773    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-istio-podinfo\") pod \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\" (UID: \"a28e0e27-b1cb-4d23-a869-a291ccfd30ae\") "
Oct 30 19:38:36.499965 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.499912    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-workload-socket" pod "a28e0e27-b1cb-4d23-a869-a291ccfd30ae" (UID: "a28e0e27-b1cb-4d23-a869-a291ccfd30ae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:36.500138 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.500085    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-credential-socket" pod "a28e0e27-b1cb-4d23-a869-a291ccfd30ae" (UID: "a28e0e27-b1cb-4d23-a869-a291ccfd30ae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:36.500214 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.500137    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-istiod-ca-cert" pod "a28e0e27-b1cb-4d23-a869-a291ccfd30ae" (UID: "a28e0e27-b1cb-4d23-a869-a291ccfd30ae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:38:36.500783 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.500758    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:36.500924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.500788    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:36.500924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.500805    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:36.500924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.500820    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:36.500924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.500834    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:36.502123 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.502097    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-kube-api-access-8nbgt" pod "a28e0e27-b1cb-4d23-a869-a291ccfd30ae" (UID: "a28e0e27-b1cb-4d23-a869-a291ccfd30ae"). InnerVolumeSpecName "kube-api-access-8nbgt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:36.502262 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.502237    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-istio-podinfo" pod "a28e0e27-b1cb-4d23-a869-a291ccfd30ae" (UID: "a28e0e27-b1cb-4d23-a869-a291ccfd30ae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:38:36.502383 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.502368    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-istio-token" pod "a28e0e27-b1cb-4d23-a869-a291ccfd30ae" (UID: "a28e0e27-b1cb-4d23-a869-a291ccfd30ae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:36.503193 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.503173    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-istio-envoy" pod "a28e0e27-b1cb-4d23-a869-a291ccfd30ae" (UID: "a28e0e27-b1cb-4d23-a869-a291ccfd30ae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:36.602441 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.602408    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:36.602441 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.602437    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:36.602699 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.602449    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nbgt\" (UniqueName: \"kubernetes.io/projected/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-kube-api-access-8nbgt\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:36.602699 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.602464    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a28e0e27-b1cb-4d23-a869-a291ccfd30ae-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:36.806469 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.806367    5907 generic.go:356] "Generic (PLEG): container finished" podID="a28e0e27-b1cb-4d23-a869-a291ccfd30ae" containerID="1070687e67289e59a279dbcf53d3a636424a458821396b18052231793cb79091" exitCode=0
Oct 30 19:38:36.806650 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.806460    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"
Oct 30 19:38:36.806650 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.806496    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz" event={"ID":"a28e0e27-b1cb-4d23-a869-a291ccfd30ae","Type":"ContainerDied","Data":"1070687e67289e59a279dbcf53d3a636424a458821396b18052231793cb79091"}
Oct 30 19:38:36.806650 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.806560    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz" event={"ID":"a28e0e27-b1cb-4d23-a869-a291ccfd30ae","Type":"ContainerDied","Data":"8c614da420d3dd3d77bd2721fedbcc257f3449a5cfae542ee56bedb218d91d94"}
Oct 30 19:38:36.806650 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.806593    5907 scope.go:122] "RemoveContainer" containerID="1070687e67289e59a279dbcf53d3a636424a458821396b18052231793cb79091"
Oct 30 19:38:36.809177 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.809156    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-46bd1516-hxjfq"
Oct 30 19:38:36.809758 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.809738    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-46bd1516-hxjfq" event={"ID":"956f186f-15d0-4f15-806e-4ad518542d35","Type":"ContainerDied","Data":"77296f885e8915aa7f1a21e85b120f0998c1b3816edbb77718a54693d812a109"}
Oct 30 19:38:36.809822 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.809784    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="77296f885e8915aa7f1a21e85b120f0998c1b3816edbb77718a54693d812a109"
Oct 30 19:38:36.820074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.820056    5907 scope.go:122] "RemoveContainer" containerID="1070687e67289e59a279dbcf53d3a636424a458821396b18052231793cb79091"
Oct 30 19:38:36.820363 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:38:36.820343    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1070687e67289e59a279dbcf53d3a636424a458821396b18052231793cb79091\": container with ID starting with 1070687e67289e59a279dbcf53d3a636424a458821396b18052231793cb79091 not found: ID does not exist" containerID="1070687e67289e59a279dbcf53d3a636424a458821396b18052231793cb79091"
Oct 30 19:38:36.820434 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.820369    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1070687e67289e59a279dbcf53d3a636424a458821396b18052231793cb79091"} err="failed to get container status \"1070687e67289e59a279dbcf53d3a636424a458821396b18052231793cb79091\": rpc error: code = NotFound desc = could not find container \"1070687e67289e59a279dbcf53d3a636424a458821396b18052231793cb79091\": container with ID starting with 1070687e67289e59a279dbcf53d3a636424a458821396b18052231793cb79091 not found: ID does not exist"
Oct 30 19:38:36.866476 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.866433    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"]
Oct 30 19:38:36.879976 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:36.879947    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-62b91eb5-openshift-default-858b7c55f7-nqfjz"]
Oct 30 19:38:38.032052 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:38.032016    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-010785bc-9prhn"]
Oct 30 19:38:38.044939 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:38.044910    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-010785bc-9prhn"]
Oct 30 19:38:38.304568 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:38.304490    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="588869d6-5b48-4d49-8c18-4d91a187c7db" path="/var/lib/kubelet/pods/588869d6-5b48-4d49-8c18-4d91a187c7db/volumes"
Oct 30 19:38:38.305010 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:38.304995    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a28e0e27-b1cb-4d23-a869-a291ccfd30ae" path="/var/lib/kubelet/pods/a28e0e27-b1cb-4d23-a869-a291ccfd30ae/volumes"
Oct 30 19:38:39.037042 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:39.037011    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1913ec02-q8sd5"]
Oct 30 19:38:39.049800 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:39.049770    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1913ec02-q8sd5"]
Oct 30 19:38:39.783443 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:39.783418    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:38:39.847192 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:39.846911    5907 generic.go:356] "Generic (PLEG): container finished" podID="ce9284b5-c1c6-4390-a5aa-410c7538e9eb" containerID="725d58d2a861352576472103d26ce61f093b5ab05c768b1d41bf8aacd7311c9c" exitCode=0
Oct 30 19:38:39.847192 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:39.846937    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"
Oct 30 19:38:39.847192 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:39.846956    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh" event={"ID":"ce9284b5-c1c6-4390-a5aa-410c7538e9eb","Type":"ContainerDied","Data":"725d58d2a861352576472103d26ce61f093b5ab05c768b1d41bf8aacd7311c9c"}
Oct 30 19:38:39.847192 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:39.847008    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh" event={"ID":"ce9284b5-c1c6-4390-a5aa-410c7538e9eb","Type":"ContainerDied","Data":"c1ab4b7d46d962bd9f4797a2374c3a1b3f724698f7cd6a465ec95db5587dd0d0"}
Oct 30 19:38:39.847192 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:39.847032    5907 scope.go:122] "RemoveContainer" containerID="725d58d2a861352576472103d26ce61f093b5ab05c768b1d41bf8aacd7311c9c"
Oct 30 19:38:39.860753 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:39.860725    5907 scope.go:122] "RemoveContainer" containerID="42591b20effe399ad712e14ff8d7ce6d7a59128a3ff52a71e3cc807d7904b742"
Oct 30 19:38:39.876156 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:39.876134    5907 scope.go:122] "RemoveContainer" containerID="725d58d2a861352576472103d26ce61f093b5ab05c768b1d41bf8aacd7311c9c"
Oct 30 19:38:39.876808 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:38:39.876778    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"725d58d2a861352576472103d26ce61f093b5ab05c768b1d41bf8aacd7311c9c\": container with ID starting with 725d58d2a861352576472103d26ce61f093b5ab05c768b1d41bf8aacd7311c9c not found: ID does not exist" containerID="725d58d2a861352576472103d26ce61f093b5ab05c768b1d41bf8aacd7311c9c"
Oct 30 19:38:39.876956 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:39.876818    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"725d58d2a861352576472103d26ce61f093b5ab05c768b1d41bf8aacd7311c9c"} err="failed to get container status \"725d58d2a861352576472103d26ce61f093b5ab05c768b1d41bf8aacd7311c9c\": rpc error: code = NotFound desc = could not find container \"725d58d2a861352576472103d26ce61f093b5ab05c768b1d41bf8aacd7311c9c\": container with ID starting with 725d58d2a861352576472103d26ce61f093b5ab05c768b1d41bf8aacd7311c9c not found: ID does not exist"
Oct 30 19:38:39.876956 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:39.876842    5907 scope.go:122] "RemoveContainer" containerID="42591b20effe399ad712e14ff8d7ce6d7a59128a3ff52a71e3cc807d7904b742"
Oct 30 19:38:39.877345 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:38:39.877313    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42591b20effe399ad712e14ff8d7ce6d7a59128a3ff52a71e3cc807d7904b742\": container with ID starting with 42591b20effe399ad712e14ff8d7ce6d7a59128a3ff52a71e3cc807d7904b742 not found: ID does not exist" containerID="42591b20effe399ad712e14ff8d7ce6d7a59128a3ff52a71e3cc807d7904b742"
Oct 30 19:38:39.877454 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:39.877352    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42591b20effe399ad712e14ff8d7ce6d7a59128a3ff52a71e3cc807d7904b742"} err="failed to get container status \"42591b20effe399ad712e14ff8d7ce6d7a59128a3ff52a71e3cc807d7904b742\": rpc error: code = NotFound desc = could not find container \"42591b20effe399ad712e14ff8d7ce6d7a59128a3ff52a71e3cc807d7904b742\": container with ID starting with 42591b20effe399ad712e14ff8d7ce6d7a59128a3ff52a71e3cc807d7904b742 not found: ID does not exist"
Oct 30 19:38:39.916192 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:39.916165    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-workload-socket\") pod \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") "
Oct 30 19:38:39.916336 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:39.916208    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-kube-api-access-cd6tk\" (UniqueName: \"kubernetes.io/projected/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-kube-api-access-cd6tk\") pod \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") "
Oct 30 19:38:39.916336 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:39.916285    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-credential-socket\") pod \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") "
Oct 30 19:38:39.916458 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:39.916410    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-istio-data\") pod \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") "
Oct 30 19:38:39.916637 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:39.916616    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-istio-envoy\") pod \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") "
Oct 30 19:38:39.916714 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:39.916658    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-istiod-ca-cert\") pod \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") "
Oct 30 19:38:39.916714 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:39.916697    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-istio-podinfo\") pod \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") "
Oct 30 19:38:39.916898 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:39.916730    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-workload-certs\") pod \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") "
Oct 30 19:38:39.916898 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:39.916776    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-credential-socket" pod "ce9284b5-c1c6-4390-a5aa-410c7538e9eb" (UID: "ce9284b5-c1c6-4390-a5aa-410c7538e9eb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:39.916898 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:39.916830    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-istio-token\" (UniqueName: \"kubernetes.io/projected/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-istio-token\") pod \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\" (UID: \"ce9284b5-c1c6-4390-a5aa-410c7538e9eb\") "
Oct 30 19:38:39.917057 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:39.916961    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-istio-data" pod "ce9284b5-c1c6-4390-a5aa-410c7538e9eb" (UID: "ce9284b5-c1c6-4390-a5aa-410c7538e9eb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:39.917248 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:39.917225    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-workload-certs" pod "ce9284b5-c1c6-4390-a5aa-410c7538e9eb" (UID: "ce9284b5-c1c6-4390-a5aa-410c7538e9eb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:39.917730 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:39.917668    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:39.917730 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:39.917694    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:39.917730 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:39.917708    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:39.917987 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:39.917917    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-istiod-ca-cert" pod "ce9284b5-c1c6-4390-a5aa-410c7538e9eb" (UID: "ce9284b5-c1c6-4390-a5aa-410c7538e9eb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:38:39.917987 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:39.917970    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-workload-socket" pod "ce9284b5-c1c6-4390-a5aa-410c7538e9eb" (UID: "ce9284b5-c1c6-4390-a5aa-410c7538e9eb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:39.920056 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:39.920030    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-kube-api-access-cd6tk" pod "ce9284b5-c1c6-4390-a5aa-410c7538e9eb" (UID: "ce9284b5-c1c6-4390-a5aa-410c7538e9eb"). InnerVolumeSpecName "kube-api-access-cd6tk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:39.920163 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:39.920125    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-istio-envoy" pod "ce9284b5-c1c6-4390-a5aa-410c7538e9eb" (UID: "ce9284b5-c1c6-4390-a5aa-410c7538e9eb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:39.920328 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:39.920311    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-istio-token" pod "ce9284b5-c1c6-4390-a5aa-410c7538e9eb" (UID: "ce9284b5-c1c6-4390-a5aa-410c7538e9eb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:39.921265 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:39.921240    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-istio-podinfo" pod "ce9284b5-c1c6-4390-a5aa-410c7538e9eb" (UID: "ce9284b5-c1c6-4390-a5aa-410c7538e9eb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:38:40.019389 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:40.019360    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:40.019522 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:40.019392    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:40.019522 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:40.019406    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:40.019522 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:40.019414    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:40.019522 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:40.019424    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:40.019522 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:40.019433    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cd6tk\" (UniqueName: \"kubernetes.io/projected/ce9284b5-c1c6-4390-a5aa-410c7538e9eb-kube-api-access-cd6tk\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:40.201879 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:40.201824    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"]
Oct 30 19:38:40.216657 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:40.216612    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-5016f41a-gw-openshift-default-75f8474db9-klrrh"]
Oct 30 19:38:40.290159 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:40.290126    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ce9284b5-c1c6-4390-a5aa-410c7538e9eb" path="/var/lib/kubelet/pods/ce9284b5-c1c6-4390-a5aa-410c7538e9eb/volumes"
Oct 30 19:38:40.290642 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:40.290627    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e346d798-b892-42b0-9832-ffb14d72f22a" path="/var/lib/kubelet/pods/e346d798-b892-42b0-9832-ffb14d72f22a/volumes"
Oct 30 19:38:42.035001 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:42.034966    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a9c1b2dc-whzhk"]
Oct 30 19:38:42.047762 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:42.047737    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a9c1b2dc-whzhk"]
Oct 30 19:38:42.293277 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:42.293196    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="af57f262-1206-4414-bbb7-7aa4aca88391" path="/var/lib/kubelet/pods/af57f262-1206-4414-bbb7-7aa4aca88391/volumes"
Oct 30 19:38:43.084671 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:43.084636    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-46bd1516-6d7b686fb-rfggk"]
Oct 30 19:38:43.120082 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:43.120055    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-46bd1516-b8686b4d7-w8mcm"]
Oct 30 19:38:45.032585 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:45.032551    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0c4a8c8d-v6k4n"]
Oct 30 19:38:45.045025 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:45.044998    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ef1c16f2-4c9d5"]
Oct 30 19:38:45.050167 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:45.050146    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0c4a8c8d-v6k4n"]
Oct 30 19:38:45.054795 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:45.054776    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ef1c16f2-4c9d5"]
Oct 30 19:38:45.986840 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:45.986799    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-46bd1516-hxjfq"]
Oct 30 19:38:45.999120 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:45.999089    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-46bd1516-hxjfq"]
Oct 30 19:38:46.299054 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:46.298976    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0be07a33-84b6-4d99-ae4e-ef83769bd359" path="/var/lib/kubelet/pods/0be07a33-84b6-4d99-ae4e-ef83769bd359/volumes"
Oct 30 19:38:46.299456 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:46.299439    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="956f186f-15d0-4f15-806e-4ad518542d35" path="/var/lib/kubelet/pods/956f186f-15d0-4f15-806e-4ad518542d35/volumes"
Oct 30 19:38:46.299903 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:46.299881    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fd223aca-6c79-4fb0-8946-c37bddd8f6be" path="/var/lib/kubelet/pods/fd223aca-6c79-4fb0-8946-c37bddd8f6be/volumes"
Oct 30 19:38:46.546760 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:46.546726    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"]
Oct 30 19:38:46.547151 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:46.547127    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw" podUID="fbdba47e-927d-454e-b860-ed6ea071304b" containerName="istio-proxy" containerID="cri-o://8e91f00589b8503459bc781a01f5c186ac5e69f9dd3a7cbc8eb87fdd9430ae67" gracePeriod=30
Oct 30 19:38:46.841082 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:46.841049    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-b9c43c35-7dc4b6fdf8-v4zvp"
Oct 30 19:38:46.848083 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:46.848045    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-b9c43c35-66d656d8f9-7qxjn"
Oct 30 19:38:47.504722 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:38:47.504686    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:38:48.011397 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.011359    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:38:48.075034 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.075001    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"]
Oct 30 19:38:48.076687 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.076654    5907 cpu_manager.go:414] "containerMap: removing container" podUID="ce9284b5-c1c6-4390-a5aa-410c7538e9eb" containerName="istio-proxy"
Oct 30 19:38:48.076687 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.076688    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ce9284b5-c1c6-4390-a5aa-410c7538e9eb" containerName="istio-proxy"
Oct 30 19:38:48.076930 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.076722    5907 cpu_manager.go:414] "containerMap: removing container" podUID="ce9284b5-c1c6-4390-a5aa-410c7538e9eb" containerName="istio-proxy"
Oct 30 19:38:48.076930 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.076732    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ce9284b5-c1c6-4390-a5aa-410c7538e9eb" containerName="istio-proxy"
Oct 30 19:38:48.076930 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.076761    5907 cpu_manager.go:414] "containerMap: removing container" podUID="a28e0e27-b1cb-4d23-a869-a291ccfd30ae" containerName="istio-proxy"
Oct 30 19:38:48.076930 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.076769    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a28e0e27-b1cb-4d23-a869-a291ccfd30ae" containerName="istio-proxy"
Oct 30 19:38:48.076930 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.076780    5907 cpu_manager.go:414] "containerMap: removing container" podUID="956f186f-15d0-4f15-806e-4ad518542d35" containerName="revoke-keys"
Oct 30 19:38:48.076930 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.076787    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="956f186f-15d0-4f15-806e-4ad518542d35" containerName="revoke-keys"
Oct 30 19:38:48.077227 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.077018    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="ce9284b5-c1c6-4390-a5aa-410c7538e9eb" containerName="istio-proxy"
Oct 30 19:38:48.077227 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.077039    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="ce9284b5-c1c6-4390-a5aa-410c7538e9eb" containerName="istio-proxy"
Oct 30 19:38:48.077227 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.077054    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="a28e0e27-b1cb-4d23-a869-a291ccfd30ae" containerName="istio-proxy"
Oct 30 19:38:48.077227 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.077065    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="956f186f-15d0-4f15-806e-4ad518542d35" containerName="revoke-keys"
Oct 30 19:38:48.085312 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.085286    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-46bd1516-6d7b686fb-rfggk" podUID="64d0a763-2322-4d30-9745-c650d15e1d05" containerName="payload-processing" containerID="cri-o://1992630853dd50061c0d8c753b411dcdcec0c7cd32a877c7ff052e9fad5c2293" gracePeriod=25
Oct 30 19:38:48.104512 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.104483    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:38:48.112788 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.112760    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"]
Oct 30 19:38:48.120697 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.120661    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-46bd1516-b8686b4d7-w8mcm" podUID="bea8608e-101a-4aac-8164-9d757d8281c9" containerName="payload-pre-processing" containerID="cri-o://859a231589d446e3d57d72672140943aeb9e22e2211b2f9b180a3ddaa8b2d629" gracePeriod=25
Oct 30 19:38:48.124701 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.124678    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-c6fd28cf-openshift-default-dockercfg-5fxgm\""
Oct 30 19:38:48.221671 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.221643    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61a95e57-7f30-4985-8efc-9cbbc0843993-workload-certs\") pod \"e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") " pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:38:48.221829 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.221791    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61a95e57-7f30-4985-8efc-9cbbc0843993-istio-token\") pod \"e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") " pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:38:48.221971 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.221953    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61a95e57-7f30-4985-8efc-9cbbc0843993-istio-podinfo\") pod \"e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") " pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:38:48.222015 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.221985    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61a95e57-7f30-4985-8efc-9cbbc0843993-istio-envoy\") pod \"e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") " pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:38:48.222050 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.222014    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61a95e57-7f30-4985-8efc-9cbbc0843993-workload-socket\") pod \"e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") " pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:38:48.222150 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.222129    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61a95e57-7f30-4985-8efc-9cbbc0843993-credential-socket\") pod \"e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") " pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:38:48.222183 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.222162    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61a95e57-7f30-4985-8efc-9cbbc0843993-istio-data\") pod \"e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") " pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:38:48.222246 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.222234    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61a95e57-7f30-4985-8efc-9cbbc0843993-istiod-ca-cert\") pod \"e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") " pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:38:48.222283 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.222269    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjlxn\" (UniqueName: \"kubernetes.io/projected/61a95e57-7f30-4985-8efc-9cbbc0843993-kube-api-access-sjlxn\") pod \"e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") " pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:38:48.539421 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.456244    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61a95e57-7f30-4985-8efc-9cbbc0843993-workload-certs\") pod \"e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") " pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:38:48.539421 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.456477    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61a95e57-7f30-4985-8efc-9cbbc0843993-istio-token\") pod \"e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") " pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:38:48.539421 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.456726    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61a95e57-7f30-4985-8efc-9cbbc0843993-istio-podinfo\") pod \"e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") " pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:38:48.539421 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.456767    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61a95e57-7f30-4985-8efc-9cbbc0843993-istio-envoy\") pod \"e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") " pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:38:48.539421 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.456807    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61a95e57-7f30-4985-8efc-9cbbc0843993-workload-socket\") pod \"e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") " pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:38:48.539421 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.456884    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61a95e57-7f30-4985-8efc-9cbbc0843993-credential-socket\") pod \"e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") " pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:38:48.539421 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.456915    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61a95e57-7f30-4985-8efc-9cbbc0843993-istio-data\") pod \"e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") " pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:38:48.539421 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.457002    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61a95e57-7f30-4985-8efc-9cbbc0843993-istiod-ca-cert\") pod \"e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") " pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:38:48.539421 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.457063    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjlxn\" (UniqueName: \"kubernetes.io/projected/61a95e57-7f30-4985-8efc-9cbbc0843993-kube-api-access-sjlxn\") pod \"e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") " pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:38:48.539421 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.458112    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61a95e57-7f30-4985-8efc-9cbbc0843993-workload-certs\") pod \"e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") " pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:38:48.539421 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.504756    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61a95e57-7f30-4985-8efc-9cbbc0843993-istio-data\") pod \"e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") " pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:38:48.539421 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.505056    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61a95e57-7f30-4985-8efc-9cbbc0843993-workload-socket\") pod \"e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") " pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:38:48.539421 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.505273    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61a95e57-7f30-4985-8efc-9cbbc0843993-credential-socket\") pod \"e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") " pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:38:48.539421 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.505847    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61a95e57-7f30-4985-8efc-9cbbc0843993-istiod-ca-cert\") pod \"e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") " pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:38:48.539421 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.506836    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61a95e57-7f30-4985-8efc-9cbbc0843993-istio-podinfo\") pod \"e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") " pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:38:48.539421 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.521047    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61a95e57-7f30-4985-8efc-9cbbc0843993-istio-envoy\") pod \"e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") " pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:38:48.539421 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.521597    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61a95e57-7f30-4985-8efc-9cbbc0843993-istio-token\") pod \"e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") " pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:38:48.591932 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.582594    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjlxn\" (UniqueName: \"kubernetes.io/projected/61a95e57-7f30-4985-8efc-9cbbc0843993-kube-api-access-sjlxn\") pod \"e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") " pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:38:48.858362 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.858275    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:38:48.882212 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.882180    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-46bd1516-6d7b686fb-rfggk"
Oct 30 19:38:49.021550 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.968204    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/64d0a763-2322-4d30-9745-c650d15e1d05-kube-api-access-kd286\" (UniqueName: \"kubernetes.io/projected/64d0a763-2322-4d30-9745-c650d15e1d05-kube-api-access-kd286\") pod \"64d0a763-2322-4d30-9745-c650d15e1d05\" (UID: \"64d0a763-2322-4d30-9745-c650d15e1d05\") "
Oct 30 19:38:49.021550 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:48.968278    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/64d0a763-2322-4d30-9745-c650d15e1d05-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64d0a763-2322-4d30-9745-c650d15e1d05-plugins-config-volume\") pod \"64d0a763-2322-4d30-9745-c650d15e1d05\" (UID: \"64d0a763-2322-4d30-9745-c650d15e1d05\") "
Oct 30 19:38:49.111005 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:49.083177    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64d0a763-2322-4d30-9745-c650d15e1d05-plugins-config-volume" pod "64d0a763-2322-4d30-9745-c650d15e1d05" (UID: "64d0a763-2322-4d30-9745-c650d15e1d05"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:38:49.125184 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:49.114054    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64d0a763-2322-4d30-9745-c650d15e1d05-kube-api-access-kd286" pod "64d0a763-2322-4d30-9745-c650d15e1d05" (UID: "64d0a763-2322-4d30-9745-c650d15e1d05"). InnerVolumeSpecName "kube-api-access-kd286". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:49.183335 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:49.165050    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kd286\" (UniqueName: \"kubernetes.io/projected/64d0a763-2322-4d30-9745-c650d15e1d05-kube-api-access-kd286\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:49.183335 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:49.165084    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64d0a763-2322-4d30-9745-c650d15e1d05-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:49.198766 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:49.198723    5907 generic.go:356] "Generic (PLEG): container finished" podID="64d0a763-2322-4d30-9745-c650d15e1d05" containerID="1992630853dd50061c0d8c753b411dcdcec0c7cd32a877c7ff052e9fad5c2293" exitCode=0
Oct 30 19:38:49.199407 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:49.199381    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-46bd1516-6d7b686fb-rfggk" event={"ID":"64d0a763-2322-4d30-9745-c650d15e1d05","Type":"ContainerDied","Data":"1992630853dd50061c0d8c753b411dcdcec0c7cd32a877c7ff052e9fad5c2293"}
Oct 30 19:38:49.199618 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:49.199593    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-46bd1516-6d7b686fb-rfggk" event={"ID":"64d0a763-2322-4d30-9745-c650d15e1d05","Type":"ContainerDied","Data":"343a862edead27e3a395297479e3a42c040e8fe240d52538ffb8e3ec69bbae13"}
Oct 30 19:38:49.199758 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:49.199745    5907 scope.go:122] "RemoveContainer" containerID="1992630853dd50061c0d8c753b411dcdcec0c7cd32a877c7ff052e9fad5c2293"
Oct 30 19:38:49.200086 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:49.200069    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-46bd1516-6d7b686fb-rfggk"
Oct 30 19:38:49.244210 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:49.243800    5907 generic.go:356] "Generic (PLEG): container finished" podID="bea8608e-101a-4aac-8164-9d757d8281c9" containerID="859a231589d446e3d57d72672140943aeb9e22e2211b2f9b180a3ddaa8b2d629" exitCode=0
Oct 30 19:38:49.244210 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:49.243970    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-46bd1516-b8686b4d7-w8mcm" event={"ID":"bea8608e-101a-4aac-8164-9d757d8281c9","Type":"ContainerDied","Data":"859a231589d446e3d57d72672140943aeb9e22e2211b2f9b180a3ddaa8b2d629"}
Oct 30 19:38:49.385580 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:49.385519    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-46bd1516-6d7b686fb-rfggk"]
Oct 30 19:38:49.389403 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:49.389374    5907 scope.go:122] "RemoveContainer" containerID="1992630853dd50061c0d8c753b411dcdcec0c7cd32a877c7ff052e9fad5c2293"
Oct 30 19:38:49.390297 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:38:49.390270    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1992630853dd50061c0d8c753b411dcdcec0c7cd32a877c7ff052e9fad5c2293\": container with ID starting with 1992630853dd50061c0d8c753b411dcdcec0c7cd32a877c7ff052e9fad5c2293 not found: ID does not exist" containerID="1992630853dd50061c0d8c753b411dcdcec0c7cd32a877c7ff052e9fad5c2293"
Oct 30 19:38:49.390444 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:49.390419    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1992630853dd50061c0d8c753b411dcdcec0c7cd32a877c7ff052e9fad5c2293"} err="failed to get container status \"1992630853dd50061c0d8c753b411dcdcec0c7cd32a877c7ff052e9fad5c2293\": rpc error: code = NotFound desc = could not find container \"1992630853dd50061c0d8c753b411dcdcec0c7cd32a877c7ff052e9fad5c2293\": container with ID starting with 1992630853dd50061c0d8c753b411dcdcec0c7cd32a877c7ff052e9fad5c2293 not found: ID does not exist"
Oct 30 19:38:49.393604 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:49.393243    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-46bd1516-b8686b4d7-w8mcm"
Oct 30 19:38:49.398576 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:49.398558    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-46bd1516-6d7b686fb-rfggk"]
Oct 30 19:38:49.474429 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:49.474389    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bea8608e-101a-4aac-8164-9d757d8281c9-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bea8608e-101a-4aac-8164-9d757d8281c9-plugins-config-volume\") pod \"bea8608e-101a-4aac-8164-9d757d8281c9\" (UID: \"bea8608e-101a-4aac-8164-9d757d8281c9\") "
Oct 30 19:38:49.474646 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:49.474463    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bea8608e-101a-4aac-8164-9d757d8281c9-kube-api-access-szzv7\" (UniqueName: \"kubernetes.io/projected/bea8608e-101a-4aac-8164-9d757d8281c9-kube-api-access-szzv7\") pod \"bea8608e-101a-4aac-8164-9d757d8281c9\" (UID: \"bea8608e-101a-4aac-8164-9d757d8281c9\") "
Oct 30 19:38:49.476410 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:49.476379    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bea8608e-101a-4aac-8164-9d757d8281c9-plugins-config-volume" pod "bea8608e-101a-4aac-8164-9d757d8281c9" (UID: "bea8608e-101a-4aac-8164-9d757d8281c9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:38:49.478611 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:49.478586    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bea8608e-101a-4aac-8164-9d757d8281c9-kube-api-access-szzv7" pod "bea8608e-101a-4aac-8164-9d757d8281c9" (UID: "bea8608e-101a-4aac-8164-9d757d8281c9"). InnerVolumeSpecName "kube-api-access-szzv7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:49.515906 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:49.515845    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"]
Oct 30 19:38:49.516352 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:38:49.516330    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61a95e57_7f30_4985_8efc_9cbbc0843993.slice/crio-9a05aaaee8c0885cfe6b0d00e8308a9994a70c2869f98139f6fb26a8ca8bd24b WatchSource:0}: Error finding container 9a05aaaee8c0885cfe6b0d00e8308a9994a70c2869f98139f6fb26a8ca8bd24b: Status 404 returned error can't find the container with id 9a05aaaee8c0885cfe6b0d00e8308a9994a70c2869f98139f6fb26a8ca8bd24b
Oct 30 19:38:49.519917 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:49.519843    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:38:49.519998 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:49.519934    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:38:49.519998 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:49.519966    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:38:49.577091 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:49.577059    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bea8608e-101a-4aac-8164-9d757d8281c9-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:49.577091 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:49.577086    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szzv7\" (UniqueName: \"kubernetes.io/projected/bea8608e-101a-4aac-8164-9d757d8281c9-kube-api-access-szzv7\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:50.033115 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:50.033085    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3eb9ce78-524x8"]
Oct 30 19:38:50.038448 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:50.038420    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3eb9ce78-524x8"]
Oct 30 19:38:50.315752 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:50.315669    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="64d0a763-2322-4d30-9745-c650d15e1d05" path="/var/lib/kubelet/pods/64d0a763-2322-4d30-9745-c650d15e1d05/volumes"
Oct 30 19:38:50.316318 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:50.316298    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ad250179-35bc-4289-9732-91a9ab45ab67" path="/var/lib/kubelet/pods/ad250179-35bc-4289-9732-91a9ab45ab67/volumes"
Oct 30 19:38:50.316687 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:50.316669    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-46bd1516-b8686b4d7-w8mcm"
Oct 30 19:38:50.317015 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:50.316991    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw" event={"ID":"61a95e57-7f30-4985-8efc-9cbbc0843993","Type":"ContainerStarted","Data":"f856dc1ad44a1d324020a8bc3c92782d6505bcffef001671948f0d800580d12a"}
Oct 30 19:38:50.317095 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:50.317040    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw" event={"ID":"61a95e57-7f30-4985-8efc-9cbbc0843993","Type":"ContainerStarted","Data":"9a05aaaee8c0885cfe6b0d00e8308a9994a70c2869f98139f6fb26a8ca8bd24b"}
Oct 30 19:38:50.317095 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:50.317072    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-46bd1516-b8686b4d7-w8mcm" event={"ID":"bea8608e-101a-4aac-8164-9d757d8281c9","Type":"ContainerDied","Data":"5bbf5d4b03e16af8e7987efe644ec5e1ef4545795f915feff4e7b76ace0d77d8"}
Oct 30 19:38:50.317232 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:50.317108    5907 scope.go:122] "RemoveContainer" containerID="859a231589d446e3d57d72672140943aeb9e22e2211b2f9b180a3ddaa8b2d629"
Oct 30 19:38:50.365430 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:50.365387    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw" podStartSLOduration=2.365372639 podStartE2EDuration="2.365372639s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:38:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:38:50.360276608 +0000 UTC m=+1042.203208548" watchObservedRunningTime="2026-10-30 19:38:50.365372639 +0000 UTC m=+1042.208304578"
Oct 30 19:38:50.403687 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:50.403661    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-46bd1516-b8686b4d7-w8mcm"]
Oct 30 19:38:50.411989 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:50.411963    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-46bd1516-b8686b4d7-w8mcm"]
Oct 30 19:38:50.811925 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:50.811848    5907 patch_prober.go:29] interesting pod/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:38:50.812330 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:50.811941    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw" podUID="fbdba47e-927d-454e-b860-ed6ea071304b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:38:50.859174 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:50.859141    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:38:50.864499 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:50.864476    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:38:51.330123 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:51.330092    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:38:51.331101 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:51.331085    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:38:51.843233 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:51.843208    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"
Oct 30 19:38:51.990252 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:51.987774    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fbdba47e-927d-454e-b860-ed6ea071304b-istio-token\" (UniqueName: \"kubernetes.io/projected/fbdba47e-927d-454e-b860-ed6ea071304b-istio-token\") pod \"fbdba47e-927d-454e-b860-ed6ea071304b\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") "
Oct 30 19:38:51.990252 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:51.987927    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fbdba47e-927d-454e-b860-ed6ea071304b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbdba47e-927d-454e-b860-ed6ea071304b-istio-podinfo\") pod \"fbdba47e-927d-454e-b860-ed6ea071304b\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") "
Oct 30 19:38:51.990252 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:51.987968    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbdba47e-927d-454e-b860-ed6ea071304b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbdba47e-927d-454e-b860-ed6ea071304b-workload-certs\") pod \"fbdba47e-927d-454e-b860-ed6ea071304b\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") "
Oct 30 19:38:51.990252 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:51.988032    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbdba47e-927d-454e-b860-ed6ea071304b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbdba47e-927d-454e-b860-ed6ea071304b-istio-data\") pod \"fbdba47e-927d-454e-b860-ed6ea071304b\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") "
Oct 30 19:38:51.990252 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:51.988073    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbdba47e-927d-454e-b860-ed6ea071304b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbdba47e-927d-454e-b860-ed6ea071304b-workload-socket\") pod \"fbdba47e-927d-454e-b860-ed6ea071304b\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") "
Oct 30 19:38:51.990252 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:51.988261    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fbdba47e-927d-454e-b860-ed6ea071304b-kube-api-access-kpk8v\" (UniqueName: \"kubernetes.io/projected/fbdba47e-927d-454e-b860-ed6ea071304b-kube-api-access-kpk8v\") pod \"fbdba47e-927d-454e-b860-ed6ea071304b\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") "
Oct 30 19:38:51.990252 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:51.988384    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fbdba47e-927d-454e-b860-ed6ea071304b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbdba47e-927d-454e-b860-ed6ea071304b-istiod-ca-cert\") pod \"fbdba47e-927d-454e-b860-ed6ea071304b\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") "
Oct 30 19:38:51.990252 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:51.988454    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbdba47e-927d-454e-b860-ed6ea071304b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbdba47e-927d-454e-b860-ed6ea071304b-credential-socket\") pod \"fbdba47e-927d-454e-b860-ed6ea071304b\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") "
Oct 30 19:38:51.990252 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:51.988545    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbdba47e-927d-454e-b860-ed6ea071304b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbdba47e-927d-454e-b860-ed6ea071304b-istio-envoy\") pod \"fbdba47e-927d-454e-b860-ed6ea071304b\" (UID: \"fbdba47e-927d-454e-b860-ed6ea071304b\") "
Oct 30 19:38:51.991389 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:51.991362    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbdba47e-927d-454e-b860-ed6ea071304b-workload-certs" pod "fbdba47e-927d-454e-b860-ed6ea071304b" (UID: "fbdba47e-927d-454e-b860-ed6ea071304b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:51.995705 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:51.995194    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbdba47e-927d-454e-b860-ed6ea071304b-credential-socket" pod "fbdba47e-927d-454e-b860-ed6ea071304b" (UID: "fbdba47e-927d-454e-b860-ed6ea071304b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:51.995705 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:51.995615    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbdba47e-927d-454e-b860-ed6ea071304b-istiod-ca-cert" pod "fbdba47e-927d-454e-b860-ed6ea071304b" (UID: "fbdba47e-927d-454e-b860-ed6ea071304b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:38:51.995705 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:51.995678    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbdba47e-927d-454e-b860-ed6ea071304b-workload-socket" pod "fbdba47e-927d-454e-b860-ed6ea071304b" (UID: "fbdba47e-927d-454e-b860-ed6ea071304b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:52.035733 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:52.035013    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbdba47e-927d-454e-b860-ed6ea071304b-istio-envoy" pod "fbdba47e-927d-454e-b860-ed6ea071304b" (UID: "fbdba47e-927d-454e-b860-ed6ea071304b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:52.039475 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:52.036617    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fbdba47e-927d-454e-b860-ed6ea071304b-istio-podinfo" pod "fbdba47e-927d-454e-b860-ed6ea071304b" (UID: "fbdba47e-927d-454e-b860-ed6ea071304b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:38:52.039475 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:52.036620    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbdba47e-927d-454e-b860-ed6ea071304b-istio-token" pod "fbdba47e-927d-454e-b860-ed6ea071304b" (UID: "fbdba47e-927d-454e-b860-ed6ea071304b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:52.039475 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:52.039110    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbdba47e-927d-454e-b860-ed6ea071304b-istio-data" pod "fbdba47e-927d-454e-b860-ed6ea071304b" (UID: "fbdba47e-927d-454e-b860-ed6ea071304b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:52.039475 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:52.039433    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbdba47e-927d-454e-b860-ed6ea071304b-kube-api-access-kpk8v" pod "fbdba47e-927d-454e-b860-ed6ea071304b" (UID: "fbdba47e-927d-454e-b860-ed6ea071304b"). InnerVolumeSpecName "kube-api-access-kpk8v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:52.091444 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:52.091406    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kpk8v\" (UniqueName: \"kubernetes.io/projected/fbdba47e-927d-454e-b860-ed6ea071304b-kube-api-access-kpk8v\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:52.091444 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:52.091436    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbdba47e-927d-454e-b860-ed6ea071304b-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:52.091444 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:52.091445    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbdba47e-927d-454e-b860-ed6ea071304b-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:52.091444 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:52.091453    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbdba47e-927d-454e-b860-ed6ea071304b-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:52.091790 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:52.091463    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbdba47e-927d-454e-b860-ed6ea071304b-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:52.091790 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:52.091472    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbdba47e-927d-454e-b860-ed6ea071304b-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:52.091790 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:52.091479    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbdba47e-927d-454e-b860-ed6ea071304b-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:52.091790 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:52.091487    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbdba47e-927d-454e-b860-ed6ea071304b-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:52.091790 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:52.091495    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbdba47e-927d-454e-b860-ed6ea071304b-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:38:52.292037 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:52.291949    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bea8608e-101a-4aac-8164-9d757d8281c9" path="/var/lib/kubelet/pods/bea8608e-101a-4aac-8164-9d757d8281c9/volumes"
Oct 30 19:38:52.344477 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:52.344440    5907 generic.go:356] "Generic (PLEG): container finished" podID="fbdba47e-927d-454e-b860-ed6ea071304b" containerID="8e91f00589b8503459bc781a01f5c186ac5e69f9dd3a7cbc8eb87fdd9430ae67" exitCode=0
Oct 30 19:38:52.344683 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:52.344515    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"
Oct 30 19:38:52.344683 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:52.344552    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw" event={"ID":"fbdba47e-927d-454e-b860-ed6ea071304b","Type":"ContainerDied","Data":"8e91f00589b8503459bc781a01f5c186ac5e69f9dd3a7cbc8eb87fdd9430ae67"}
Oct 30 19:38:52.344683 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:52.344637    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw" event={"ID":"fbdba47e-927d-454e-b860-ed6ea071304b","Type":"ContainerDied","Data":"0ab66b26b47f44fac1bbe34528c99b880fbe33438b3391d97228d704986956d6"}
Oct 30 19:38:52.344683 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:52.344664    5907 scope.go:122] "RemoveContainer" containerID="8e91f00589b8503459bc781a01f5c186ac5e69f9dd3a7cbc8eb87fdd9430ae67"
Oct 30 19:38:52.356790 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:52.356766    5907 scope.go:122] "RemoveContainer" containerID="8e91f00589b8503459bc781a01f5c186ac5e69f9dd3a7cbc8eb87fdd9430ae67"
Oct 30 19:38:52.357103 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:38:52.357080    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e91f00589b8503459bc781a01f5c186ac5e69f9dd3a7cbc8eb87fdd9430ae67\": container with ID starting with 8e91f00589b8503459bc781a01f5c186ac5e69f9dd3a7cbc8eb87fdd9430ae67 not found: ID does not exist" containerID="8e91f00589b8503459bc781a01f5c186ac5e69f9dd3a7cbc8eb87fdd9430ae67"
Oct 30 19:38:52.357159 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:52.357113    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e91f00589b8503459bc781a01f5c186ac5e69f9dd3a7cbc8eb87fdd9430ae67"} err="failed to get container status \"8e91f00589b8503459bc781a01f5c186ac5e69f9dd3a7cbc8eb87fdd9430ae67\": rpc error: code = NotFound desc = could not find container \"8e91f00589b8503459bc781a01f5c186ac5e69f9dd3a7cbc8eb87fdd9430ae67\": container with ID starting with 8e91f00589b8503459bc781a01f5c186ac5e69f9dd3a7cbc8eb87fdd9430ae67 not found: ID does not exist"
Oct 30 19:38:52.402748 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:52.402715    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"]
Oct 30 19:38:52.418169 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:52.418130    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-46bd1516-openshift-default-574c796f87-dg8fw"]
Oct 30 19:38:54.290800 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:54.290768    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fbdba47e-927d-454e-b860-ed6ea071304b" path="/var/lib/kubelet/pods/fbdba47e-927d-454e-b860-ed6ea071304b/volumes"
Oct 30 19:38:54.607438 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:54.607336    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c6fd28cf-6f49bf84bd-j4mh6"]
Oct 30 19:38:54.608257 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:54.608235    5907 cpu_manager.go:414] "containerMap: removing container" podUID="64d0a763-2322-4d30-9745-c650d15e1d05" containerName="payload-processing"
Oct 30 19:38:54.608257 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:54.608257    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="64d0a763-2322-4d30-9745-c650d15e1d05" containerName="payload-processing"
Oct 30 19:38:54.608438 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:54.608291    5907 cpu_manager.go:414] "containerMap: removing container" podUID="fbdba47e-927d-454e-b860-ed6ea071304b" containerName="istio-proxy"
Oct 30 19:38:54.608438 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:54.608298    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fbdba47e-927d-454e-b860-ed6ea071304b" containerName="istio-proxy"
Oct 30 19:38:54.608438 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:54.608309    5907 cpu_manager.go:414] "containerMap: removing container" podUID="bea8608e-101a-4aac-8164-9d757d8281c9" containerName="payload-pre-processing"
Oct 30 19:38:54.608438 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:54.608317    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bea8608e-101a-4aac-8164-9d757d8281c9" containerName="payload-pre-processing"
Oct 30 19:38:54.608642 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:54.608518    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="64d0a763-2322-4d30-9745-c650d15e1d05" containerName="payload-processing"
Oct 30 19:38:54.608642 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:54.608539    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="fbdba47e-927d-454e-b860-ed6ea071304b" containerName="istio-proxy"
Oct 30 19:38:54.608642 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:54.608548    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="bea8608e-101a-4aac-8164-9d757d8281c9" containerName="payload-pre-processing"
Oct 30 19:38:54.614128 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:54.614109    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c6fd28cf-6f49bf84bd-j4mh6"
Oct 30 19:38:54.635833 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:54.635800    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c6fd28cf-6f49bf84bd-j4mh6"]
Oct 30 19:38:54.641019 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:54.640998    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-c6fd28cf\""
Oct 30 19:38:54.641019 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:54.641006    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-c6fd28cf-dockercfg-4czhx\""
Oct 30 19:38:54.658742 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:54.658716    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c6fd28cf-667f98f88c-kstvs"]
Oct 30 19:38:54.664161 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:54.664142    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6fd28cf-667f98f88c-kstvs"
Oct 30 19:38:54.680166 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:54.680141    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c6fd28cf-667f98f88c-kstvs"]
Oct 30 19:38:54.894682 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:54.889168    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04b9bd1e-b30d-4d4f-b045-9441ce2415f7-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c6fd28cf-667f98f88c-kstvs\" (UID: \"04b9bd1e-b30d-4d4f-b045-9441ce2415f7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c6fd28cf-667f98f88c-kstvs"
Oct 30 19:38:54.894682 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:54.889315    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx4tb\" (UniqueName: \"kubernetes.io/projected/04b9bd1e-b30d-4d4f-b045-9441ce2415f7-kube-api-access-nx4tb\") pod \"payload-pre-processing-e2e-mt-c6fd28cf-667f98f88c-kstvs\" (UID: \"04b9bd1e-b30d-4d4f-b045-9441ce2415f7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c6fd28cf-667f98f88c-kstvs"
Oct 30 19:38:54.894682 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:54.889419    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m426j\" (UniqueName: \"kubernetes.io/projected/5e15145e-9cae-4aba-9ba4-9fb31522485e-kube-api-access-m426j\") pod \"payload-processing-e2e-mt-c6fd28cf-6f49bf84bd-j4mh6\" (UID: \"5e15145e-9cae-4aba-9ba4-9fb31522485e\") " pod="openshift-ingress/payload-processing-e2e-mt-c6fd28cf-6f49bf84bd-j4mh6"
Oct 30 19:38:54.894682 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:54.889532    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e15145e-9cae-4aba-9ba4-9fb31522485e-plugins-config-volume\") pod \"payload-processing-e2e-mt-c6fd28cf-6f49bf84bd-j4mh6\" (UID: \"5e15145e-9cae-4aba-9ba4-9fb31522485e\") " pod="openshift-ingress/payload-processing-e2e-mt-c6fd28cf-6f49bf84bd-j4mh6"
Oct 30 19:38:55.097181 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:55.091740    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m426j\" (UniqueName: \"kubernetes.io/projected/5e15145e-9cae-4aba-9ba4-9fb31522485e-kube-api-access-m426j\") pod \"payload-processing-e2e-mt-c6fd28cf-6f49bf84bd-j4mh6\" (UID: \"5e15145e-9cae-4aba-9ba4-9fb31522485e\") " pod="openshift-ingress/payload-processing-e2e-mt-c6fd28cf-6f49bf84bd-j4mh6"
Oct 30 19:38:55.097181 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:55.092005    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e15145e-9cae-4aba-9ba4-9fb31522485e-plugins-config-volume\") pod \"payload-processing-e2e-mt-c6fd28cf-6f49bf84bd-j4mh6\" (UID: \"5e15145e-9cae-4aba-9ba4-9fb31522485e\") " pod="openshift-ingress/payload-processing-e2e-mt-c6fd28cf-6f49bf84bd-j4mh6"
Oct 30 19:38:55.097181 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:55.092149    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04b9bd1e-b30d-4d4f-b045-9441ce2415f7-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c6fd28cf-667f98f88c-kstvs\" (UID: \"04b9bd1e-b30d-4d4f-b045-9441ce2415f7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c6fd28cf-667f98f88c-kstvs"
Oct 30 19:38:55.097181 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:55.092320    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx4tb\" (UniqueName: \"kubernetes.io/projected/04b9bd1e-b30d-4d4f-b045-9441ce2415f7-kube-api-access-nx4tb\") pod \"payload-pre-processing-e2e-mt-c6fd28cf-667f98f88c-kstvs\" (UID: \"04b9bd1e-b30d-4d4f-b045-9441ce2415f7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c6fd28cf-667f98f88c-kstvs"
Oct 30 19:38:55.097181 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:55.096099    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e15145e-9cae-4aba-9ba4-9fb31522485e-plugins-config-volume\") pod \"payload-processing-e2e-mt-c6fd28cf-6f49bf84bd-j4mh6\" (UID: \"5e15145e-9cae-4aba-9ba4-9fb31522485e\") " pod="openshift-ingress/payload-processing-e2e-mt-c6fd28cf-6f49bf84bd-j4mh6"
Oct 30 19:38:55.097181 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:55.096795    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04b9bd1e-b30d-4d4f-b045-9441ce2415f7-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c6fd28cf-667f98f88c-kstvs\" (UID: \"04b9bd1e-b30d-4d4f-b045-9441ce2415f7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c6fd28cf-667f98f88c-kstvs"
Oct 30 19:38:55.139885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:55.132637    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx4tb\" (UniqueName: \"kubernetes.io/projected/04b9bd1e-b30d-4d4f-b045-9441ce2415f7-kube-api-access-nx4tb\") pod \"payload-pre-processing-e2e-mt-c6fd28cf-667f98f88c-kstvs\" (UID: \"04b9bd1e-b30d-4d4f-b045-9441ce2415f7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c6fd28cf-667f98f88c-kstvs"
Oct 30 19:38:55.171986 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:55.169582    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m426j\" (UniqueName: \"kubernetes.io/projected/5e15145e-9cae-4aba-9ba4-9fb31522485e-kube-api-access-m426j\") pod \"payload-processing-e2e-mt-c6fd28cf-6f49bf84bd-j4mh6\" (UID: \"5e15145e-9cae-4aba-9ba4-9fb31522485e\") " pod="openshift-ingress/payload-processing-e2e-mt-c6fd28cf-6f49bf84bd-j4mh6"
Oct 30 19:38:55.199931 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:55.192313    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7a76fa6b-75gxz"]
Oct 30 19:38:55.220419 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:55.220385    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7a76fa6b-75gxz"]
Oct 30 19:38:55.229588 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:55.229564    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c6fd28cf-6f49bf84bd-j4mh6"
Oct 30 19:38:55.281913 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:55.281836    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6fd28cf-667f98f88c-kstvs"
Oct 30 19:38:55.551425 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:38:55.551390    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e15145e_9cae_4aba_9ba4_9fb31522485e.slice/crio-b6fa54f0111789560040089ee84ea81fbe1382c0e67cf92e3bb01da8880cf6ec WatchSource:0}: Error finding container b6fa54f0111789560040089ee84ea81fbe1382c0e67cf92e3bb01da8880cf6ec: Status 404 returned error can't find the container with id b6fa54f0111789560040089ee84ea81fbe1382c0e67cf92e3bb01da8880cf6ec
Oct 30 19:38:55.553076 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:55.553049    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c6fd28cf-6f49bf84bd-j4mh6"]
Oct 30 19:38:55.762311 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:55.762272    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c6fd28cf-667f98f88c-kstvs"]
Oct 30 19:38:56.029631 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:56.029601    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e8596f5d-6kbzh"]
Oct 30 19:38:56.034586 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:56.034562    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e8596f5d-6kbzh"]
Oct 30 19:38:56.290012 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:56.289980    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6e93fb18-a78b-458c-a4a9-c47d88f7ac4d" path="/var/lib/kubelet/pods/6e93fb18-a78b-458c-a4a9-c47d88f7ac4d/volumes"
Oct 30 19:38:56.290485 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:56.290471    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ddf77163-3370-4a29-a119-abe7e1723841" path="/var/lib/kubelet/pods/ddf77163-3370-4a29-a119-abe7e1723841/volumes"
Oct 30 19:38:56.392703 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:56.392665    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6fd28cf-667f98f88c-kstvs" event={"ID":"04b9bd1e-b30d-4d4f-b045-9441ce2415f7","Type":"ContainerStarted","Data":"3784b140c5972dc88af50e5259a98ca0b4920517bf9cd461e0082f96591adeda"}
Oct 30 19:38:56.392952 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:56.392726    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6fd28cf-667f98f88c-kstvs" event={"ID":"04b9bd1e-b30d-4d4f-b045-9441ce2415f7","Type":"ContainerStarted","Data":"614ab8de7d825ae7486d3b81b9eaec1606577e6aeecfc0de1f8893db792e7af1"}
Oct 30 19:38:56.392952 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:56.392755    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6fd28cf-667f98f88c-kstvs"
Oct 30 19:38:56.394503 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:56.394471    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c6fd28cf-6f49bf84bd-j4mh6" event={"ID":"5e15145e-9cae-4aba-9ba4-9fb31522485e","Type":"ContainerStarted","Data":"d604bff502ec51978ce7d483266602b6605ba702ad06444eeb1ffb376365c0fc"}
Oct 30 19:38:56.394630 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:56.394527    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c6fd28cf-6f49bf84bd-j4mh6" event={"ID":"5e15145e-9cae-4aba-9ba4-9fb31522485e","Type":"ContainerStarted","Data":"b6fa54f0111789560040089ee84ea81fbe1382c0e67cf92e3bb01da8880cf6ec"}
Oct 30 19:38:56.394630 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:56.394555    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-c6fd28cf-6f49bf84bd-j4mh6"
Oct 30 19:38:56.437370 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:56.437319    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6fd28cf-667f98f88c-kstvs" podStartSLOduration=2.437305813 podStartE2EDuration="2.437305813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:38:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:38:56.434908776 +0000 UTC m=+1048.277840712" watchObservedRunningTime="2026-10-30 19:38:56.437305813 +0000 UTC m=+1048.280237805"
Oct 30 19:38:56.481766 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:38:56.481716    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-c6fd28cf-6f49bf84bd-j4mh6" podStartSLOduration=2.481703773 podStartE2EDuration="2.481703773s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:38:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:38:56.476402963 +0000 UTC m=+1048.319334901" watchObservedRunningTime="2026-10-30 19:38:56.481703773 +0000 UTC m=+1048.324635721"
Oct 30 19:39:01.030652 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:01.030619    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7a9bf31c-knzts"]
Oct 30 19:39:01.043986 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:01.043960    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7a9bf31c-knzts"]
Oct 30 19:39:02.291769 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:02.291736    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927" path="/var/lib/kubelet/pods/cfb8f2ab-9cc6-4a0b-9ca2-20e5d34b2927/volumes"
Oct 30 19:39:07.406311 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:07.406285    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-c6fd28cf-6f49bf84bd-j4mh6"
Oct 30 19:39:07.406789 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:07.406352    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6fd28cf-667f98f88c-kstvs"
Oct 30 19:39:07.736758 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:07.736681    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b9c43c35-4bc9r"]
Oct 30 19:39:07.742767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:07.742747    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b9c43c35-4bc9r"
Oct 30 19:39:07.882070 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:07.873745    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:39:07.884199 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:07.882777    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:39:07.891643 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:07.891586    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b9c43c35-4bc9r"]
Oct 30 19:39:07.970755 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:07.970719    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccjkx\" (UniqueName: \"kubernetes.io/projected/119d34bf-b0f0-47cd-923c-a7cadf41aa05-kube-api-access-ccjkx\") pod \"maas-api-revoke-keys-e2e-ait-b9c43c35-4bc9r\" (UID: \"119d34bf-b0f0-47cd-923c-a7cadf41aa05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b9c43c35-4bc9r"
Oct 30 19:39:07.970945 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:07.970791    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/119d34bf-b0f0-47cd-923c-a7cadf41aa05-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-b9c43c35-4bc9r\" (UID: \"119d34bf-b0f0-47cd-923c-a7cadf41aa05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b9c43c35-4bc9r"
Oct 30 19:39:07.970945 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:07.970915    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/119d34bf-b0f0-47cd-923c-a7cadf41aa05-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-b9c43c35-4bc9r\" (UID: \"119d34bf-b0f0-47cd-923c-a7cadf41aa05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b9c43c35-4bc9r"
Oct 30 19:39:08.072760 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:08.072675    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccjkx\" (UniqueName: \"kubernetes.io/projected/119d34bf-b0f0-47cd-923c-a7cadf41aa05-kube-api-access-ccjkx\") pod \"maas-api-revoke-keys-e2e-ait-b9c43c35-4bc9r\" (UID: \"119d34bf-b0f0-47cd-923c-a7cadf41aa05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b9c43c35-4bc9r"
Oct 30 19:39:08.072760 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:08.072754    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/119d34bf-b0f0-47cd-923c-a7cadf41aa05-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-b9c43c35-4bc9r\" (UID: \"119d34bf-b0f0-47cd-923c-a7cadf41aa05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b9c43c35-4bc9r"
Oct 30 19:39:08.072970 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:08.072818    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/119d34bf-b0f0-47cd-923c-a7cadf41aa05-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-b9c43c35-4bc9r\" (UID: \"119d34bf-b0f0-47cd-923c-a7cadf41aa05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b9c43c35-4bc9r"
Oct 30 19:39:08.073432 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:08.073410    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/119d34bf-b0f0-47cd-923c-a7cadf41aa05-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-b9c43c35-4bc9r\" (UID: \"119d34bf-b0f0-47cd-923c-a7cadf41aa05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b9c43c35-4bc9r"
Oct 30 19:39:08.075446 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:08.075420    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/119d34bf-b0f0-47cd-923c-a7cadf41aa05-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-b9c43c35-4bc9r\" (UID: \"119d34bf-b0f0-47cd-923c-a7cadf41aa05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b9c43c35-4bc9r"
Oct 30 19:39:08.093213 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:08.093191    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccjkx\" (UniqueName: \"kubernetes.io/projected/119d34bf-b0f0-47cd-923c-a7cadf41aa05-kube-api-access-ccjkx\") pod \"maas-api-revoke-keys-e2e-ait-b9c43c35-4bc9r\" (UID: \"119d34bf-b0f0-47cd-923c-a7cadf41aa05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b9c43c35-4bc9r"
Oct 30 19:39:08.211555 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:08.211250    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b9c43c35-4bc9r"
Oct 30 19:39:08.694075 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:39:08.694041    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod119d34bf_b0f0_47cd_923c_a7cadf41aa05.slice/crio-482221ee79663c664b64b7492dd7998136e83c6957707906d80d4e93ac66c511 WatchSource:0}: Error finding container 482221ee79663c664b64b7492dd7998136e83c6957707906d80d4e93ac66c511: Status 404 returned error can't find the container with id 482221ee79663c664b64b7492dd7998136e83c6957707906d80d4e93ac66c511
Oct 30 19:39:08.695564 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:08.695532    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b9c43c35-4bc9r"]
Oct 30 19:39:09.689384 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:09.689325    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b9c43c35-4bc9r" event={"ID":"119d34bf-b0f0-47cd-923c-a7cadf41aa05","Type":"ContainerStarted","Data":"8ab75d85fd7dd1932fe55faa0d97064d72c19956df816c68eb0bad55fd1d399c"}
Oct 30 19:39:09.689720 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:09.689693    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b9c43c35-4bc9r" event={"ID":"119d34bf-b0f0-47cd-923c-a7cadf41aa05","Type":"ContainerStarted","Data":"482221ee79663c664b64b7492dd7998136e83c6957707906d80d4e93ac66c511"}
Oct 30 19:39:10.026541 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:10.026440    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b9c43c35-4bc9r" podStartSLOduration=3.026426271 podStartE2EDuration="3.026426271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:39:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:39:09.755292661 +0000 UTC m=+1061.598224600" watchObservedRunningTime="2026-10-30 19:39:10.026426271 +0000 UTC m=+1061.869358253"
Oct 30 19:39:10.030974 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:10.030943    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6753d129-czr6b"]
Oct 30 19:39:10.036237 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:10.036213    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6753d129-czr6b"]
Oct 30 19:39:10.290959 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:10.290876    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3821123c-0cb7-40dc-989c-c0333c6d1e7b" path="/var/lib/kubelet/pods/3821123c-0cb7-40dc-989c-c0333c6d1e7b/volumes"
Oct 30 19:39:10.703960 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:10.703921    5907 generic.go:356] "Generic (PLEG): container finished" podID="119d34bf-b0f0-47cd-923c-a7cadf41aa05" containerID="8ab75d85fd7dd1932fe55faa0d97064d72c19956df816c68eb0bad55fd1d399c" exitCode=0
Oct 30 19:39:10.704153 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:10.704005    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b9c43c35-4bc9r" event={"ID":"119d34bf-b0f0-47cd-923c-a7cadf41aa05","Type":"ContainerDied","Data":"8ab75d85fd7dd1932fe55faa0d97064d72c19956df816c68eb0bad55fd1d399c"}
Oct 30 19:39:11.888657 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:11.888633    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b9c43c35-4bc9r"
Oct 30 19:39:12.046279 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:12.046193    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/119d34bf-b0f0-47cd-923c-a7cadf41aa05-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/119d34bf-b0f0-47cd-923c-a7cadf41aa05-maas-api-xks-ca\") pod \"119d34bf-b0f0-47cd-923c-a7cadf41aa05\" (UID: \"119d34bf-b0f0-47cd-923c-a7cadf41aa05\") "
Oct 30 19:39:12.046460 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:12.046328    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/119d34bf-b0f0-47cd-923c-a7cadf41aa05-kube-api-access-ccjkx\" (UniqueName: \"kubernetes.io/projected/119d34bf-b0f0-47cd-923c-a7cadf41aa05-kube-api-access-ccjkx\") pod \"119d34bf-b0f0-47cd-923c-a7cadf41aa05\" (UID: \"119d34bf-b0f0-47cd-923c-a7cadf41aa05\") "
Oct 30 19:39:12.046540 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:12.046470    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/119d34bf-b0f0-47cd-923c-a7cadf41aa05-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/119d34bf-b0f0-47cd-923c-a7cadf41aa05-maas-api-openshift-ca\") pod \"119d34bf-b0f0-47cd-923c-a7cadf41aa05\" (UID: \"119d34bf-b0f0-47cd-923c-a7cadf41aa05\") "
Oct 30 19:39:12.046822 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:12.046792    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/119d34bf-b0f0-47cd-923c-a7cadf41aa05-maas-api-openshift-ca" pod "119d34bf-b0f0-47cd-923c-a7cadf41aa05" (UID: "119d34bf-b0f0-47cd-923c-a7cadf41aa05"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:39:12.047334 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:12.047314    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/119d34bf-b0f0-47cd-923c-a7cadf41aa05-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:39:12.048628 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:12.048601    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/119d34bf-b0f0-47cd-923c-a7cadf41aa05-maas-api-xks-ca" pod "119d34bf-b0f0-47cd-923c-a7cadf41aa05" (UID: "119d34bf-b0f0-47cd-923c-a7cadf41aa05"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:39:12.048733 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:12.048673    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/119d34bf-b0f0-47cd-923c-a7cadf41aa05-kube-api-access-ccjkx" pod "119d34bf-b0f0-47cd-923c-a7cadf41aa05" (UID: "119d34bf-b0f0-47cd-923c-a7cadf41aa05"). InnerVolumeSpecName "kube-api-access-ccjkx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:39:12.148945 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:12.148899    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/119d34bf-b0f0-47cd-923c-a7cadf41aa05-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:39:12.148945 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:12.148940    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ccjkx\" (UniqueName: \"kubernetes.io/projected/119d34bf-b0f0-47cd-923c-a7cadf41aa05-kube-api-access-ccjkx\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:39:12.725218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:12.725135    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b9c43c35-4bc9r"
Oct 30 19:39:12.725218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:12.725175    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b9c43c35-4bc9r" event={"ID":"119d34bf-b0f0-47cd-923c-a7cadf41aa05","Type":"ContainerDied","Data":"482221ee79663c664b64b7492dd7998136e83c6957707906d80d4e93ac66c511"}
Oct 30 19:39:12.725417 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:12.725226    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="482221ee79663c664b64b7492dd7998136e83c6957707906d80d4e93ac66c511"
Oct 30 19:39:15.048000 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:15.047967    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-24ad9216-vcvqp"]
Oct 30 19:39:15.060782 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:15.060753    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-24ad9216-vcvqp"]
Oct 30 19:39:16.290748 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:16.290719    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="49804909-841f-4a00-abe1-8316ee7aa58f" path="/var/lib/kubelet/pods/49804909-841f-4a00-abe1-8316ee7aa58f/volumes"
Oct 30 19:39:17.502992 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:39:17.502955    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:39:18.144248 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:18.144214    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b9c43c35-7dc4b6fdf8-v4zvp"]
Oct 30 19:39:18.163443 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:18.163412    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b9c43c35-66d656d8f9-7qxjn"]
Oct 30 19:39:19.590278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:19.590208    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b9c43c35-4bc9r"]
Oct 30 19:39:19.648157 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:19.648120    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b9c43c35-4bc9r"]
Oct 30 19:39:20.292978 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:20.292944    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="119d34bf-b0f0-47cd-923c-a7cadf41aa05" path="/var/lib/kubelet/pods/119d34bf-b0f0-47cd-923c-a7cadf41aa05/volumes"
Oct 30 19:39:21.137991 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:21.137954    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"]
Oct 30 19:39:21.138504 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:21.138190    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r" podUID="c8399c42-686d-4164-b96d-e61f6a99c8bd" containerName="istio-proxy" containerID="cri-o://befb5c826eb7e3be2ed79a72442967a477fa7247a3b0c7cf1b2da0c628e98c36" gracePeriod=30
Oct 30 19:39:22.538721 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:22.538682    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"]
Oct 30 19:39:22.593497 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:22.593387    5907 cpu_manager.go:414] "containerMap: removing container" podUID="119d34bf-b0f0-47cd-923c-a7cadf41aa05" containerName="revoke-keys"
Oct 30 19:39:22.593497 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:22.593432    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="119d34bf-b0f0-47cd-923c-a7cadf41aa05" containerName="revoke-keys"
Oct 30 19:39:22.593831 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:22.593761    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="119d34bf-b0f0-47cd-923c-a7cadf41aa05" containerName="revoke-keys"
Oct 30 19:39:22.623513 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:22.623483    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"]
Oct 30 19:39:22.627385 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:22.627363    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:39:22.646080 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:22.646051    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w5-3c89af-openshift-default-dockercfg-l9lmn\""
Oct 30 19:39:22.698537 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:22.698504    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-istio-token\") pod \"e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") " pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:39:22.698701 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:22.698548    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-workload-socket\") pod \"e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") " pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:39:22.698755 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:22.698734    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-istio-podinfo\") pod \"e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") " pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:39:22.698824 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:22.698807    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-istio-data\") pod \"e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") " pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:39:22.698956 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:22.698943    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-credential-socket\") pod \"e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") " pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:39:22.699073 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:22.699061    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-workload-certs\") pod \"e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") " pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:39:22.699119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:22.699082    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-istiod-ca-cert\") pod \"e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") " pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:39:22.699157 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:22.699134    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8kvj\" (UniqueName: \"kubernetes.io/projected/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-kube-api-access-t8kvj\") pod \"e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") " pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:39:22.699191 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:22.699156    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-istio-envoy\") pod \"e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") " pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:39:22.811989 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:22.811553    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-credential-socket\") pod \"e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") " pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:39:22.811989 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:22.811793    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-workload-certs\") pod \"e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") " pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:39:22.811989 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:22.811831    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-istiod-ca-cert\") pod \"e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") " pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:39:22.811989 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:22.811907    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8kvj\" (UniqueName: \"kubernetes.io/projected/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-kube-api-access-t8kvj\") pod \"e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") " pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:39:22.811989 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:22.811968    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-istio-envoy\") pod \"e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") " pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:39:22.812467 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:22.812095    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-istio-token\") pod \"e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") " pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:39:22.812467 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:22.812147    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-workload-socket\") pod \"e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") " pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:39:22.812467 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:22.812358    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-istio-podinfo\") pod \"e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") " pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:39:22.812655 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:22.812492    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-istio-data\") pod \"e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") " pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:39:22.817345 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:22.817316    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-istio-data\") pod \"e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") " pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:39:22.835491 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:22.835458    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-credential-socket\") pod \"e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") " pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:39:22.836215 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:22.836184    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-workload-certs\") pod \"e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") " pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:39:22.837515 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:22.837439    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-workload-socket\") pod \"e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") " pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:39:22.851284 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:22.851148    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-istiod-ca-cert\") pod \"e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") " pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:39:22.869633 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:22.869597    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-istio-envoy\") pod \"e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") " pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:39:22.893885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:22.871353    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-istio-podinfo\") pod \"e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") " pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:39:22.905888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:22.901634    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-istio-token\") pod \"e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") " pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:39:23.015467 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:23.002635    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8kvj\" (UniqueName: \"kubernetes.io/projected/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-kube-api-access-t8kvj\") pod \"e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") " pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:39:23.146888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:23.145950    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-b9c43c35-7dc4b6fdf8-v4zvp" podUID="f39a79fa-188d-451c-af60-7a69d653d136" containerName="payload-processing" containerID="cri-o://a6ca102d4b7091c9ea905acf6da685e1b1c0e11d31efe9efaa7d4e1735f7f4ba" gracePeriod=25
Oct 30 19:39:23.165775 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:23.165035    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-b9c43c35-66d656d8f9-7qxjn" podUID="de43dd5e-6749-475c-b56e-c2fdba78aba6" containerName="payload-pre-processing" containerID="cri-o://c2d3fa15829fca941d95f266c0df27a78c23fe081af032809bafad180ba0a832" gracePeriod=25
Oct 30 19:39:23.253523 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:23.253425    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:39:23.996343 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:23.996307    5907 generic.go:356] "Generic (PLEG): container finished" podID="f39a79fa-188d-451c-af60-7a69d653d136" containerID="a6ca102d4b7091c9ea905acf6da685e1b1c0e11d31efe9efaa7d4e1735f7f4ba" exitCode=0
Oct 30 19:39:23.997173 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:23.996391    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b9c43c35-7dc4b6fdf8-v4zvp" event={"ID":"f39a79fa-188d-451c-af60-7a69d653d136","Type":"ContainerDied","Data":"a6ca102d4b7091c9ea905acf6da685e1b1c0e11d31efe9efaa7d4e1735f7f4ba"}
Oct 30 19:39:23.998880 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:23.998837    5907 generic.go:356] "Generic (PLEG): container finished" podID="de43dd5e-6749-475c-b56e-c2fdba78aba6" containerID="c2d3fa15829fca941d95f266c0df27a78c23fe081af032809bafad180ba0a832" exitCode=0
Oct 30 19:39:23.999111 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:23.998907    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b9c43c35-66d656d8f9-7qxjn" event={"ID":"de43dd5e-6749-475c-b56e-c2fdba78aba6","Type":"ContainerDied","Data":"c2d3fa15829fca941d95f266c0df27a78c23fe081af032809bafad180ba0a832"}
Oct 30 19:39:24.080734 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:24.080706    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b9c43c35-7dc4b6fdf8-v4zvp"
Oct 30 19:39:24.085235 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:24.085212    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b9c43c35-66d656d8f9-7qxjn"
Oct 30 19:39:24.142490 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:24.119516    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"]
Oct 30 19:39:24.150422 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:24.150376    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:39:24.150567 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:24.150473    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:39:24.150567 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:24.150521    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:39:24.255102 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:24.255076    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/de43dd5e-6749-475c-b56e-c2fdba78aba6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de43dd5e-6749-475c-b56e-c2fdba78aba6-plugins-config-volume\") pod \"de43dd5e-6749-475c-b56e-c2fdba78aba6\" (UID: \"de43dd5e-6749-475c-b56e-c2fdba78aba6\") "
Oct 30 19:39:24.255287 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:24.255274    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f39a79fa-188d-451c-af60-7a69d653d136-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f39a79fa-188d-451c-af60-7a69d653d136-plugins-config-volume\") pod \"f39a79fa-188d-451c-af60-7a69d653d136\" (UID: \"f39a79fa-188d-451c-af60-7a69d653d136\") "
Oct 30 19:39:24.255353 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:24.255302    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/de43dd5e-6749-475c-b56e-c2fdba78aba6-kube-api-access-jhz69\" (UniqueName: \"kubernetes.io/projected/de43dd5e-6749-475c-b56e-c2fdba78aba6-kube-api-access-jhz69\") pod \"de43dd5e-6749-475c-b56e-c2fdba78aba6\" (UID: \"de43dd5e-6749-475c-b56e-c2fdba78aba6\") "
Oct 30 19:39:24.255409 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:24.255397    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f39a79fa-188d-451c-af60-7a69d653d136-kube-api-access-vd2vq\" (UniqueName: \"kubernetes.io/projected/f39a79fa-188d-451c-af60-7a69d653d136-kube-api-access-vd2vq\") pod \"f39a79fa-188d-451c-af60-7a69d653d136\" (UID: \"f39a79fa-188d-451c-af60-7a69d653d136\") "
Oct 30 19:39:24.255631 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:24.255494    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de43dd5e-6749-475c-b56e-c2fdba78aba6-plugins-config-volume" pod "de43dd5e-6749-475c-b56e-c2fdba78aba6" (UID: "de43dd5e-6749-475c-b56e-c2fdba78aba6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:39:24.255728 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:24.255662    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f39a79fa-188d-451c-af60-7a69d653d136-plugins-config-volume" pod "f39a79fa-188d-451c-af60-7a69d653d136" (UID: "f39a79fa-188d-451c-af60-7a69d653d136"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:39:24.256415 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:24.256392    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f39a79fa-188d-451c-af60-7a69d653d136-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:39:24.256415 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:24.256414    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de43dd5e-6749-475c-b56e-c2fdba78aba6-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:39:24.257610 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:24.257583    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f39a79fa-188d-451c-af60-7a69d653d136-kube-api-access-vd2vq" pod "f39a79fa-188d-451c-af60-7a69d653d136" (UID: "f39a79fa-188d-451c-af60-7a69d653d136"). InnerVolumeSpecName "kube-api-access-vd2vq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:39:24.257700 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:24.257663    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de43dd5e-6749-475c-b56e-c2fdba78aba6-kube-api-access-jhz69" pod "de43dd5e-6749-475c-b56e-c2fdba78aba6" (UID: "de43dd5e-6749-475c-b56e-c2fdba78aba6"). InnerVolumeSpecName "kube-api-access-jhz69". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:39:24.358591 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:24.358508    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhz69\" (UniqueName: \"kubernetes.io/projected/de43dd5e-6749-475c-b56e-c2fdba78aba6-kube-api-access-jhz69\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:39:24.358591 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:24.358541    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vd2vq\" (UniqueName: \"kubernetes.io/projected/f39a79fa-188d-451c-af60-7a69d653d136-kube-api-access-vd2vq\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:39:25.142003 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:25.141824    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b9c43c35-7dc4b6fdf8-v4zvp" event={"ID":"f39a79fa-188d-451c-af60-7a69d653d136","Type":"ContainerDied","Data":"d816be2bf7f83eaa95f3e8b76c012a99e2eaf719045885c02793353bb9699bcb"}
Oct 30 19:39:25.142558 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:25.142537    5907 scope.go:122] "RemoveContainer" containerID="a6ca102d4b7091c9ea905acf6da685e1b1c0e11d31efe9efaa7d4e1735f7f4ba"
Oct 30 19:39:25.142933 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:25.142911    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b9c43c35-7dc4b6fdf8-v4zvp"
Oct 30 19:39:25.212343 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:25.212305    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b9c43c35-66d656d8f9-7qxjn"
Oct 30 19:39:25.218445 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:25.212509    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b9c43c35-66d656d8f9-7qxjn" event={"ID":"de43dd5e-6749-475c-b56e-c2fdba78aba6","Type":"ContainerDied","Data":"69f658fc17cd5d5a443d533cb39701be7388e94507780b08a31b1cf985960936"}
Oct 30 19:39:25.276114 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:25.273732    5907 scope.go:122] "RemoveContainer" containerID="c2d3fa15829fca941d95f266c0df27a78c23fe081af032809bafad180ba0a832"
Oct 30 19:39:25.312174 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:25.312030    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp" event={"ID":"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4","Type":"ContainerStarted","Data":"148ea3c66d89c893e746c8d7bda4d25f4ca0c29824aaa93c6389a86d90bca3be"}
Oct 30 19:39:25.312174 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:25.312107    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp" event={"ID":"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4","Type":"ContainerStarted","Data":"3068626d2ec4b4a0eda5e6a5eed6fcd79933e5e4106905fd7c1ebc122ff51dca"}
Oct 30 19:39:25.389683 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:25.389647    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b9c43c35-7dc4b6fdf8-v4zvp"]
Oct 30 19:39:25.401893 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:25.401077    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b9c43c35-7dc4b6fdf8-v4zvp"]
Oct 30 19:39:25.409389 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:25.408916    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b9c43c35-66d656d8f9-7qxjn"]
Oct 30 19:39:25.427951 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:25.420616    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b9c43c35-66d656d8f9-7qxjn"]
Oct 30 19:39:25.521782 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:25.521720    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp" podStartSLOduration=3.521701379 podStartE2EDuration="3.521701379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:39:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:39:25.469425823 +0000 UTC m=+1077.312357763" watchObservedRunningTime="2026-10-30 19:39:25.521701379 +0000 UTC m=+1077.364633319"
Oct 30 19:39:26.254005 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.253972    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:39:26.259491 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.259454    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:39:26.292515 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.292480    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="de43dd5e-6749-475c-b56e-c2fdba78aba6" path="/var/lib/kubelet/pods/de43dd5e-6749-475c-b56e-c2fdba78aba6/volumes"
Oct 30 19:39:26.293265 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.293244    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f39a79fa-188d-451c-af60-7a69d653d136" path="/var/lib/kubelet/pods/f39a79fa-188d-451c-af60-7a69d653d136/volumes"
Oct 30 19:39:26.354000 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.353967    5907 generic.go:356] "Generic (PLEG): container finished" podID="c8399c42-686d-4164-b96d-e61f6a99c8bd" containerID="befb5c826eb7e3be2ed79a72442967a477fa7247a3b0c7cf1b2da0c628e98c36" exitCode=0
Oct 30 19:39:26.354178 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.354039    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r" event={"ID":"c8399c42-686d-4164-b96d-e61f6a99c8bd","Type":"ContainerDied","Data":"befb5c826eb7e3be2ed79a72442967a477fa7247a3b0c7cf1b2da0c628e98c36"}
Oct 30 19:39:26.356600 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.356578    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:39:26.357619 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.357600    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:39:26.568407 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.568378    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"
Oct 30 19:39:26.766346 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.766262    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c8399c42-686d-4164-b96d-e61f6a99c8bd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8399c42-686d-4164-b96d-e61f6a99c8bd-workload-certs\") pod \"c8399c42-686d-4164-b96d-e61f6a99c8bd\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") "
Oct 30 19:39:26.766346 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.766326    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c8399c42-686d-4164-b96d-e61f6a99c8bd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8399c42-686d-4164-b96d-e61f6a99c8bd-istio-data\") pod \"c8399c42-686d-4164-b96d-e61f6a99c8bd\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") "
Oct 30 19:39:26.766346 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.766348    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c8399c42-686d-4164-b96d-e61f6a99c8bd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8399c42-686d-4164-b96d-e61f6a99c8bd-istio-podinfo\") pod \"c8399c42-686d-4164-b96d-e61f6a99c8bd\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") "
Oct 30 19:39:26.766638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.766446    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c8399c42-686d-4164-b96d-e61f6a99c8bd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8399c42-686d-4164-b96d-e61f6a99c8bd-istiod-ca-cert\") pod \"c8399c42-686d-4164-b96d-e61f6a99c8bd\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") "
Oct 30 19:39:26.766638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.766479    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c8399c42-686d-4164-b96d-e61f6a99c8bd-istio-token\" (UniqueName: \"kubernetes.io/projected/c8399c42-686d-4164-b96d-e61f6a99c8bd-istio-token\") pod \"c8399c42-686d-4164-b96d-e61f6a99c8bd\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") "
Oct 30 19:39:26.766638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.766579    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c8399c42-686d-4164-b96d-e61f6a99c8bd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8399c42-686d-4164-b96d-e61f6a99c8bd-istio-envoy\") pod \"c8399c42-686d-4164-b96d-e61f6a99c8bd\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") "
Oct 30 19:39:26.766638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.766598    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8399c42-686d-4164-b96d-e61f6a99c8bd-workload-certs" pod "c8399c42-686d-4164-b96d-e61f6a99c8bd" (UID: "c8399c42-686d-4164-b96d-e61f6a99c8bd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:39:26.766638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.766613    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8399c42-686d-4164-b96d-e61f6a99c8bd-istio-data" pod "c8399c42-686d-4164-b96d-e61f6a99c8bd" (UID: "c8399c42-686d-4164-b96d-e61f6a99c8bd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:39:26.766823 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.766746    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c8399c42-686d-4164-b96d-e61f6a99c8bd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8399c42-686d-4164-b96d-e61f6a99c8bd-workload-socket\") pod \"c8399c42-686d-4164-b96d-e61f6a99c8bd\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") "
Oct 30 19:39:26.766823 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.766787    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c8399c42-686d-4164-b96d-e61f6a99c8bd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8399c42-686d-4164-b96d-e61f6a99c8bd-credential-socket\") pod \"c8399c42-686d-4164-b96d-e61f6a99c8bd\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") "
Oct 30 19:39:26.766967 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.766841    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c8399c42-686d-4164-b96d-e61f6a99c8bd-kube-api-access-n8ldg\" (UniqueName: \"kubernetes.io/projected/c8399c42-686d-4164-b96d-e61f6a99c8bd-kube-api-access-n8ldg\") pod \"c8399c42-686d-4164-b96d-e61f6a99c8bd\" (UID: \"c8399c42-686d-4164-b96d-e61f6a99c8bd\") "
Oct 30 19:39:26.766967 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.766884    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8399c42-686d-4164-b96d-e61f6a99c8bd-istiod-ca-cert" pod "c8399c42-686d-4164-b96d-e61f6a99c8bd" (UID: "c8399c42-686d-4164-b96d-e61f6a99c8bd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:39:26.767074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.767002    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8399c42-686d-4164-b96d-e61f6a99c8bd-workload-socket" pod "c8399c42-686d-4164-b96d-e61f6a99c8bd" (UID: "c8399c42-686d-4164-b96d-e61f6a99c8bd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:39:26.767127 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.767090    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8399c42-686d-4164-b96d-e61f6a99c8bd-credential-socket" pod "c8399c42-686d-4164-b96d-e61f6a99c8bd" (UID: "c8399c42-686d-4164-b96d-e61f6a99c8bd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:39:26.767970 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.767945    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8399c42-686d-4164-b96d-e61f6a99c8bd-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:39:26.768102 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.767972    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8399c42-686d-4164-b96d-e61f6a99c8bd-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:39:26.768102 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.767988    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8399c42-686d-4164-b96d-e61f6a99c8bd-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:39:26.768102 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.768003    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8399c42-686d-4164-b96d-e61f6a99c8bd-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:39:26.768102 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.768018    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8399c42-686d-4164-b96d-e61f6a99c8bd-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:39:26.768913 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.768888    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8399c42-686d-4164-b96d-e61f6a99c8bd-istio-token" pod "c8399c42-686d-4164-b96d-e61f6a99c8bd" (UID: "c8399c42-686d-4164-b96d-e61f6a99c8bd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:39:26.769134 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.769115    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8399c42-686d-4164-b96d-e61f6a99c8bd-kube-api-access-n8ldg" pod "c8399c42-686d-4164-b96d-e61f6a99c8bd" (UID: "c8399c42-686d-4164-b96d-e61f6a99c8bd"). InnerVolumeSpecName "kube-api-access-n8ldg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:39:26.769219 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.769203    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8399c42-686d-4164-b96d-e61f6a99c8bd-istio-envoy" pod "c8399c42-686d-4164-b96d-e61f6a99c8bd" (UID: "c8399c42-686d-4164-b96d-e61f6a99c8bd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:39:26.769265 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.769252    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c8399c42-686d-4164-b96d-e61f6a99c8bd-istio-podinfo" pod "c8399c42-686d-4164-b96d-e61f6a99c8bd" (UID: "c8399c42-686d-4164-b96d-e61f6a99c8bd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:39:26.869974 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.869940    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8399c42-686d-4164-b96d-e61f6a99c8bd-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:39:26.869974 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.869973    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8399c42-686d-4164-b96d-e61f6a99c8bd-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:39:26.869974 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.869983    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8399c42-686d-4164-b96d-e61f6a99c8bd-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:39:26.870225 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:26.869992    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8ldg\" (UniqueName: \"kubernetes.io/projected/c8399c42-686d-4164-b96d-e61f6a99c8bd-kube-api-access-n8ldg\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:39:27.368225 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:27.368194    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"
Oct 30 19:39:27.368706 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:27.368222    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r" event={"ID":"c8399c42-686d-4164-b96d-e61f6a99c8bd","Type":"ContainerDied","Data":"43366102b6b3d6e34fc06b2d18ba459f675122016df9cc39a01f308546eea025"}
Oct 30 19:39:27.368706 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:27.368288    5907 scope.go:122] "RemoveContainer" containerID="befb5c826eb7e3be2ed79a72442967a477fa7247a3b0c7cf1b2da0c628e98c36"
Oct 30 19:39:27.459069 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:27.459035    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"]
Oct 30 19:39:27.463977 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:27.463951    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-b9c43c35-openshift-default-bb6bbd885-f8n6r"]
Oct 30 19:39:28.699895 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:28.696470    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c8399c42-686d-4164-b96d-e61f6a99c8bd" path="/var/lib/kubelet/pods/c8399c42-686d-4164-b96d-e61f6a99c8bd/volumes"
Oct 30 19:39:29.176266 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:29.176150    5907 scope.go:122] "RemoveContainer" containerID="989698a1c05d0495ba62a009c4c96c67b6dcd6de57fb1986c4df1fdf5e3d6131"
Oct 30 19:39:29.188652 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:29.188629    5907 scope.go:122] "RemoveContainer" containerID="c31f301ba02082d628a0a9c39ff158c5ebe5f553db5f4846aabe8e71d59a0bd2"
Oct 30 19:39:29.200308 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:29.200286    5907 scope.go:122] "RemoveContainer" containerID="3835a91f398e61112d95a6c3b8d902409a01afeacd203f95cf158072da0dec40"
Oct 30 19:39:29.235118 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:29.235088    5907 scope.go:122] "RemoveContainer" containerID="722df4ac79655eb0f8b944ff920041264e09143ab09dfa0ea7cfb0c7b2db8096"
Oct 30 19:39:29.246911 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:29.246886    5907 scope.go:122] "RemoveContainer" containerID="bcfe545ac371653562bf23ba6af1c0118d245c7ffcd5bc3a34ed39fc1a0b6b58"
Oct 30 19:39:29.257451 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:29.257430    5907 scope.go:122] "RemoveContainer" containerID="1b34151b154231ecd8b9c2d1e4d685eb2cd68ad0c027d1de9a0fbab932f5f2b7"
Oct 30 19:39:29.267577 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:29.267555    5907 scope.go:122] "RemoveContainer" containerID="1a43838a9b9076a089c568c2c558d5c400bd5c869905bf242827c7f3f4acb2c6"
Oct 30 19:39:29.277603 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:29.277585    5907 scope.go:122] "RemoveContainer" containerID="8da103078d7cd3f790b1c6fc7416de7d6a99fc93b9fb24f09266fb11e896f91e"
Oct 30 19:39:29.288275 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:29.288256    5907 scope.go:122] "RemoveContainer" containerID="f3cd177963dc538a0834d5722cd6da98e883f34508829135f595fa34dbe89232"
Oct 30 19:39:29.298841 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:29.298823    5907 scope.go:122] "RemoveContainer" containerID="cf34a30ed65b866c6697a3b8feb6ccb616de32cf4ca9f455d8e0e24baf762560"
Oct 30 19:39:29.309311 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:29.309291    5907 scope.go:122] "RemoveContainer" containerID="859982c47df0da966e7079ed61d40491099540c991c1706c27c2c1d3fa23cdbf"
Oct 30 19:39:30.905980 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:30.905947    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w5-3c89af-f9bd48d78-wvb77"]
Oct 30 19:39:30.906719 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:30.906704    5907 cpu_manager.go:414] "containerMap: removing container" podUID="de43dd5e-6749-475c-b56e-c2fdba78aba6" containerName="payload-pre-processing"
Oct 30 19:39:30.906761 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:30.906720    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="de43dd5e-6749-475c-b56e-c2fdba78aba6" containerName="payload-pre-processing"
Oct 30 19:39:30.906761 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:30.906735    5907 cpu_manager.go:414] "containerMap: removing container" podUID="f39a79fa-188d-451c-af60-7a69d653d136" containerName="payload-processing"
Oct 30 19:39:30.906761 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:30.906740    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f39a79fa-188d-451c-af60-7a69d653d136" containerName="payload-processing"
Oct 30 19:39:30.906851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:30.906775    5907 cpu_manager.go:414] "containerMap: removing container" podUID="c8399c42-686d-4164-b96d-e61f6a99c8bd" containerName="istio-proxy"
Oct 30 19:39:30.906851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:30.906781    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c8399c42-686d-4164-b96d-e61f6a99c8bd" containerName="istio-proxy"
Oct 30 19:39:30.906939 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:30.906898    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="f39a79fa-188d-451c-af60-7a69d653d136" containerName="payload-processing"
Oct 30 19:39:30.906939 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:30.906917    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="c8399c42-686d-4164-b96d-e61f6a99c8bd" containerName="istio-proxy"
Oct 30 19:39:30.906939 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:30.906929    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="de43dd5e-6749-475c-b56e-c2fdba78aba6" containerName="payload-pre-processing"
Oct 30 19:39:30.914762 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:30.914741    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3c89af-f9bd48d78-wvb77"
Oct 30 19:39:30.930192 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:30.930168    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w5-3c89af-f9bd48d78-wvb77"]
Oct 30 19:39:30.936667 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:30.936643    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w5-3c89af\""
Oct 30 19:39:30.936667 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:30.936664    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w5-3c89af-dockercfg-llq4m\""
Oct 30 19:39:30.960173 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:30.960134    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3c89af-6d87768f7f-qxrpq"]
Oct 30 19:39:31.001895 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:30.997108    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3c89af-6d87768f7f-qxrpq"
Oct 30 19:39:31.088899 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:31.080338    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wq4vx\" (UniqueName: \"kubernetes.io/projected/486e38b1-f36d-48b9-ad1f-5ac88f641c49-kube-api-access-wq4vx\") pod \"payload-pre-processing-e2e-shared-a-w5-3c89af-6d87768f7f-qxrpq\" (UID: \"486e38b1-f36d-48b9-ad1f-5ac88f641c49\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3c89af-6d87768f7f-qxrpq"
Oct 30 19:39:31.088899 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:31.080465    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shnxh\" (UniqueName: \"kubernetes.io/projected/70810131-ad32-4a06-b69c-6142147dc2d9-kube-api-access-shnxh\") pod \"payload-processing-e2e-shared-a-w5-3c89af-f9bd48d78-wvb77\" (UID: \"70810131-ad32-4a06-b69c-6142147dc2d9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3c89af-f9bd48d78-wvb77"
Oct 30 19:39:31.088899 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:31.080652    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70810131-ad32-4a06-b69c-6142147dc2d9-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w5-3c89af-f9bd48d78-wvb77\" (UID: \"70810131-ad32-4a06-b69c-6142147dc2d9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3c89af-f9bd48d78-wvb77"
Oct 30 19:39:31.088899 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:31.085980    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/486e38b1-f36d-48b9-ad1f-5ac88f641c49-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w5-3c89af-6d87768f7f-qxrpq\" (UID: \"486e38b1-f36d-48b9-ad1f-5ac88f641c49\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3c89af-6d87768f7f-qxrpq"
Oct 30 19:39:31.094531 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:31.094439    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3c89af-6d87768f7f-qxrpq"]
Oct 30 19:39:31.317731 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:31.317634    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq4vx\" (UniqueName: \"kubernetes.io/projected/486e38b1-f36d-48b9-ad1f-5ac88f641c49-kube-api-access-wq4vx\") pod \"payload-pre-processing-e2e-shared-a-w5-3c89af-6d87768f7f-qxrpq\" (UID: \"486e38b1-f36d-48b9-ad1f-5ac88f641c49\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3c89af-6d87768f7f-qxrpq"
Oct 30 19:39:31.320061 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:31.320028    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shnxh\" (UniqueName: \"kubernetes.io/projected/70810131-ad32-4a06-b69c-6142147dc2d9-kube-api-access-shnxh\") pod \"payload-processing-e2e-shared-a-w5-3c89af-f9bd48d78-wvb77\" (UID: \"70810131-ad32-4a06-b69c-6142147dc2d9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3c89af-f9bd48d78-wvb77"
Oct 30 19:39:31.320462 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:31.320441    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70810131-ad32-4a06-b69c-6142147dc2d9-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w5-3c89af-f9bd48d78-wvb77\" (UID: \"70810131-ad32-4a06-b69c-6142147dc2d9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3c89af-f9bd48d78-wvb77"
Oct 30 19:39:31.320812 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:31.320795    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/486e38b1-f36d-48b9-ad1f-5ac88f641c49-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w5-3c89af-6d87768f7f-qxrpq\" (UID: \"486e38b1-f36d-48b9-ad1f-5ac88f641c49\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3c89af-6d87768f7f-qxrpq"
Oct 30 19:39:31.322536 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:31.322511    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/486e38b1-f36d-48b9-ad1f-5ac88f641c49-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w5-3c89af-6d87768f7f-qxrpq\" (UID: \"486e38b1-f36d-48b9-ad1f-5ac88f641c49\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3c89af-6d87768f7f-qxrpq"
Oct 30 19:39:31.406995 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:31.406955    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70810131-ad32-4a06-b69c-6142147dc2d9-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w5-3c89af-f9bd48d78-wvb77\" (UID: \"70810131-ad32-4a06-b69c-6142147dc2d9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3c89af-f9bd48d78-wvb77"
Oct 30 19:39:31.455551 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:31.455510    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wq4vx\" (UniqueName: \"kubernetes.io/projected/486e38b1-f36d-48b9-ad1f-5ac88f641c49-kube-api-access-wq4vx\") pod \"payload-pre-processing-e2e-shared-a-w5-3c89af-6d87768f7f-qxrpq\" (UID: \"486e38b1-f36d-48b9-ad1f-5ac88f641c49\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3c89af-6d87768f7f-qxrpq"
Oct 30 19:39:31.462036 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:31.462001    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shnxh\" (UniqueName: \"kubernetes.io/projected/70810131-ad32-4a06-b69c-6142147dc2d9-kube-api-access-shnxh\") pod \"payload-processing-e2e-shared-a-w5-3c89af-f9bd48d78-wvb77\" (UID: \"70810131-ad32-4a06-b69c-6142147dc2d9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3c89af-f9bd48d78-wvb77"
Oct 30 19:39:31.533314 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:31.533283    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3c89af-f9bd48d78-wvb77"
Oct 30 19:39:31.540991 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:31.540958    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3c89af-6d87768f7f-qxrpq"
Oct 30 19:39:31.936470 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:31.936443    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w5-3c89af-f9bd48d78-wvb77"]
Oct 30 19:39:31.958283 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:31.958232    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3c89af-6d87768f7f-qxrpq"]
Oct 30 19:39:31.965723 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:39:31.965681    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod486e38b1_f36d_48b9_ad1f_5ac88f641c49.slice/crio-2844e32e4c1136a036ed232ea47d075346a619e634ae10a045fac3cdb1fda49c WatchSource:0}: Error finding container 2844e32e4c1136a036ed232ea47d075346a619e634ae10a045fac3cdb1fda49c: Status 404 returned error can't find the container with id 2844e32e4c1136a036ed232ea47d075346a619e634ae10a045fac3cdb1fda49c
Oct 30 19:39:32.843060 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:32.843001    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3c89af-6d87768f7f-qxrpq" event={"ID":"486e38b1-f36d-48b9-ad1f-5ac88f641c49","Type":"ContainerStarted","Data":"8db1de37c378554f7b340a84d1da1e820932e5e62af9e21e076d99b9fa1778f9"}
Oct 30 19:39:32.843060 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:32.843075    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3c89af-6d87768f7f-qxrpq" event={"ID":"486e38b1-f36d-48b9-ad1f-5ac88f641c49","Type":"ContainerStarted","Data":"2844e32e4c1136a036ed232ea47d075346a619e634ae10a045fac3cdb1fda49c"}
Oct 30 19:39:32.843342 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:32.843101    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3c89af-6d87768f7f-qxrpq"
Oct 30 19:39:32.845096 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:32.845067    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3c89af-f9bd48d78-wvb77" event={"ID":"70810131-ad32-4a06-b69c-6142147dc2d9","Type":"ContainerStarted","Data":"8c25860adf6a56b7b65fcc5d17fe92151a534d689e6adac22c2a72417d2c4e4b"}
Oct 30 19:39:32.845214 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:32.845119    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3c89af-f9bd48d78-wvb77" event={"ID":"70810131-ad32-4a06-b69c-6142147dc2d9","Type":"ContainerStarted","Data":"6620365f8d751405f98931bd9ef91a45500baaaf1fdf460ab70cdaeb8f452305"}
Oct 30 19:39:32.845270 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:32.845212    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3c89af-f9bd48d78-wvb77"
Oct 30 19:39:32.886325 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:32.886268    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3c89af-6d87768f7f-qxrpq" podStartSLOduration=2.886249904 podStartE2EDuration="2.886249904s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:39:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:39:32.884675606 +0000 UTC m=+1084.727607544" watchObservedRunningTime="2026-10-30 19:39:32.886249904 +0000 UTC m=+1084.729181845"
Oct 30 19:39:32.929370 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:32.929324    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3c89af-f9bd48d78-wvb77" podStartSLOduration=2.929309716 podStartE2EDuration="2.929309716s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:39:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:39:32.926357331 +0000 UTC m=+1084.769289271" watchObservedRunningTime="2026-10-30 19:39:32.929309716 +0000 UTC m=+1084.772241656"
Oct 30 19:39:35.878708 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:35.878674    5907 generic.go:356] "Generic (PLEG): container finished" podID="61a95e57-7f30-4985-8efc-9cbbc0843993" containerID="f856dc1ad44a1d324020a8bc3c92782d6505bcffef001671948f0d800580d12a" exitCode=0
Oct 30 19:39:35.879164 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:35.878746    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw" event={"ID":"61a95e57-7f30-4985-8efc-9cbbc0843993","Type":"ContainerDied","Data":"f856dc1ad44a1d324020a8bc3c92782d6505bcffef001671948f0d800580d12a"}
Oct 30 19:39:35.879257 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:35.879242    5907 scope.go:122] "RemoveContainer" containerID="f856dc1ad44a1d324020a8bc3c92782d6505bcffef001671948f0d800580d12a"
Oct 30 19:39:35.880851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:35.880818    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:39:35.880963 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:35.880915    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:39:35.880963 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:35.880953    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:39:36.330628 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:36.330551    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:39:36.890433 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:36.890395    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw" event={"ID":"61a95e57-7f30-4985-8efc-9cbbc0843993","Type":"ContainerStarted","Data":"28b18ab7383989113aadfd005d80ea891749b0449b6592e79c7e4efaf1ccac6f"}
Oct 30 19:39:37.859345 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:37.859307    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:39:37.860302 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:37.860271    5907 patch_prober.go:29] interesting pod/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused" start-of-body=
Oct 30 19:39:37.860430 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:37.860340    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw" podUID="61a95e57-7f30-4985-8efc-9cbbc0843993" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused"
Oct 30 19:39:38.864446 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:38.864344    5907 patch_prober.go:29] interesting pod/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused" start-of-body=
Oct 30 19:39:38.865199 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:38.865161    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw" podUID="61a95e57-7f30-4985-8efc-9cbbc0843993" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused"
Oct 30 19:39:39.859948 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:39.859903    5907 patch_prober.go:29] interesting pod/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused" start-of-body=
Oct 30 19:39:39.860175 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:39.859998    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw" podUID="61a95e57-7f30-4985-8efc-9cbbc0843993" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused"
Oct 30 19:39:40.860234 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:40.860194    5907 patch_prober.go:29] interesting pod/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused" start-of-body=
Oct 30 19:39:40.860690 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:40.860245    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw" podUID="61a95e57-7f30-4985-8efc-9cbbc0843993" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused"
Oct 30 19:39:41.862309 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:41.862273    5907 patch_prober.go:29] interesting pod/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused" start-of-body=
Oct 30 19:39:41.862950 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:41.862919    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw" podUID="61a95e57-7f30-4985-8efc-9cbbc0843993" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused"
Oct 30 19:39:42.864041 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:42.864009    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:39:42.864491 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:42.864477    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:39:42.865260 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:42.865240    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:39:43.861448 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:43.861418    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3c89af-6d87768f7f-qxrpq"
Oct 30 19:39:43.861657 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:43.861494    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3c89af-f9bd48d78-wvb77"
Oct 30 19:39:47.504776 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:39:47.504729    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:39:57.041449 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:57.041407    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-127a30e6-2rrkh"]
Oct 30 19:39:57.052705 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:57.052675    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-127a30e6-2rrkh"]
Oct 30 19:39:58.289902 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:58.289848    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8e6b567c-c205-42cb-bb15-a55eb30c9921" path="/var/lib/kubelet/pods/8e6b567c-c205-42cb-bb15-a55eb30c9921/volumes"
Oct 30 19:39:59.676705 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:39:59.676657    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:40:04.445532 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:04.445470    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"]
Oct 30 19:40:04.454227 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:04.453308    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:40:04.475639 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:04.473934    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"]
Oct 30 19:40:04.475639 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:04.473972    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w5-e9de60-openshift-default-dockercfg-6tts9\""
Oct 30 19:40:04.551810 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:04.551766    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-istio-token\") pod \"e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") " pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:40:04.552132 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:04.552106    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-istiod-ca-cert\") pod \"e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") " pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:40:04.552406 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:04.552390    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-istio-podinfo\") pod \"e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") " pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:40:04.552529 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:04.552515    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-workload-socket\") pod \"e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") " pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:40:04.552716 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:04.552699    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-workload-certs\") pod \"e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") " pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:40:04.552913 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:04.552887    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-istio-data\") pod \"e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") " pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:40:04.553049 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:04.553035    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-istio-envoy\") pod \"e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") " pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:40:04.553175 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:04.553159    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-credential-socket\") pod \"e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") " pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:40:04.553337 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:04.553318    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74jdc\" (UniqueName: \"kubernetes.io/projected/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-kube-api-access-74jdc\") pod \"e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") " pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:40:04.660704 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:04.660630    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-workload-certs\") pod \"e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") " pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:40:04.708177 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:04.708093    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-workload-certs\") pod \"e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") " pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:40:04.708598 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:04.708571    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-istio-data\") pod \"e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") " pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:40:04.709184 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:04.709162    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-istio-envoy\") pod \"e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") " pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:40:04.709757 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:04.709738    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-credential-socket\") pod \"e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") " pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:40:04.710301 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:04.710284    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74jdc\" (UniqueName: \"kubernetes.io/projected/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-kube-api-access-74jdc\") pod \"e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") " pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:40:04.711312 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:04.711288    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-istio-token\") pod \"e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") " pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:40:04.711746 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:04.711727    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-istiod-ca-cert\") pod \"e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") " pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:40:04.712803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:04.712782    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-istio-podinfo\") pod \"e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") " pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:40:04.713421 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:04.713401    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-workload-socket\") pod \"e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") " pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:40:04.714098 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:04.710172    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-credential-socket\") pod \"e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") " pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:40:04.727803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:04.709083    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-istio-data\") pod \"e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") " pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:40:04.728044 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:04.713845    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-workload-socket\") pod \"e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") " pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:40:04.728153 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:04.712555    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-istiod-ca-cert\") pod \"e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") " pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:40:04.766830 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:04.766793    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74jdc\" (UniqueName: \"kubernetes.io/projected/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-kube-api-access-74jdc\") pod \"e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") " pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:40:04.773238 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:04.773206    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-istio-envoy\") pod \"e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") " pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:40:04.778900 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:04.777460    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-istio-podinfo\") pod \"e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") " pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:40:04.810600 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:04.810559    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-istio-token\") pod \"e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") " pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:40:04.811956 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:04.810907    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:40:05.567250 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:40:05.567221    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec8c99fb_671c_4aea_b30e_543bc0e3d6c1.slice/crio-0f58c4b18b51544f3ee0f03522c1037cd75611c35cc616f7ca39ea3859209cc8 WatchSource:0}: Error finding container 0f58c4b18b51544f3ee0f03522c1037cd75611c35cc616f7ca39ea3859209cc8: Status 404 returned error can't find the container with id 0f58c4b18b51544f3ee0f03522c1037cd75611c35cc616f7ca39ea3859209cc8
Oct 30 19:40:05.568158 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:05.568136    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"]
Oct 30 19:40:05.570394 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:05.570358    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:40:05.570525 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:05.570427    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:40:05.570525 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:05.570458    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:40:06.565980 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:06.565940    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9" event={"ID":"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1","Type":"ContainerStarted","Data":"b37856698cdd9aa3a02cf54ddf8eaacf36ce6bd93693522ce0733e0b689dda9e"}
Oct 30 19:40:06.566327 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:06.566002    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9" event={"ID":"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1","Type":"ContainerStarted","Data":"0f58c4b18b51544f3ee0f03522c1037cd75611c35cc616f7ca39ea3859209cc8"}
Oct 30 19:40:06.816173 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:06.816079    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:40:06.880235 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:06.880202    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:40:06.988107 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:06.988034    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9" podStartSLOduration=2.988010953 podStartE2EDuration="2.988010953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:40:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:40:06.654715665 +0000 UTC m=+1118.497647605" watchObservedRunningTime="2026-10-30 19:40:06.988010953 +0000 UTC m=+1118.830942895"
Oct 30 19:40:07.578938 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:07.578908    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:40:07.580000 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:07.579965    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:40:10.964729 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:10.964695    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w5-e9de60-75896ff6bc-vw4jn"]
Oct 30 19:40:10.971361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:10.971333    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-e9de60-75896ff6bc-vw4jn"
Oct 30 19:40:11.003812 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:11.003781    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w5-e9de60-dockercfg-4zl8j\""
Oct 30 19:40:11.004210 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:11.004168    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w5-e9de60\""
Oct 30 19:40:11.012880 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:11.012833    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w5-e9de60-75896ff6bc-vw4jn"]
Oct 30 19:40:11.195082 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:11.175535    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d9225589-5hss5"]
Oct 30 19:40:11.195082 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:11.177354    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efc8d0b8-a0b1-4c98-be74-87a9fcde2b96-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w5-e9de60-75896ff6bc-vw4jn\" (UID: \"efc8d0b8-a0b1-4c98-be74-87a9fcde2b96\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-e9de60-75896ff6bc-vw4jn"
Oct 30 19:40:11.195082 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:11.177426    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5kdq\" (UniqueName: \"kubernetes.io/projected/efc8d0b8-a0b1-4c98-be74-87a9fcde2b96-kube-api-access-p5kdq\") pod \"payload-pre-processing-e2e-shared-b-w5-e9de60-75896ff6bc-vw4jn\" (UID: \"efc8d0b8-a0b1-4c98-be74-87a9fcde2b96\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-e9de60-75896ff6bc-vw4jn"
Oct 30 19:40:11.322944 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:11.286990    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5kdq\" (UniqueName: \"kubernetes.io/projected/efc8d0b8-a0b1-4c98-be74-87a9fcde2b96-kube-api-access-p5kdq\") pod \"payload-pre-processing-e2e-shared-b-w5-e9de60-75896ff6bc-vw4jn\" (UID: \"efc8d0b8-a0b1-4c98-be74-87a9fcde2b96\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-e9de60-75896ff6bc-vw4jn"
Oct 30 19:40:11.356378 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:11.330411    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efc8d0b8-a0b1-4c98-be74-87a9fcde2b96-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w5-e9de60-75896ff6bc-vw4jn\" (UID: \"efc8d0b8-a0b1-4c98-be74-87a9fcde2b96\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-e9de60-75896ff6bc-vw4jn"
Oct 30 19:40:11.356378 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:11.331532    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efc8d0b8-a0b1-4c98-be74-87a9fcde2b96-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w5-e9de60-75896ff6bc-vw4jn\" (UID: \"efc8d0b8-a0b1-4c98-be74-87a9fcde2b96\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-e9de60-75896ff6bc-vw4jn"
Oct 30 19:40:11.356378 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:11.355345    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d9225589-5hss5"]
Oct 30 19:40:11.361919 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:11.361424    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w5-e9de60-85ffddb78-6zw8w"]
Oct 30 19:40:11.485927 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:11.485895    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5kdq\" (UniqueName: \"kubernetes.io/projected/efc8d0b8-a0b1-4c98-be74-87a9fcde2b96-kube-api-access-p5kdq\") pod \"payload-pre-processing-e2e-shared-b-w5-e9de60-75896ff6bc-vw4jn\" (UID: \"efc8d0b8-a0b1-4c98-be74-87a9fcde2b96\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-e9de60-75896ff6bc-vw4jn"
Oct 30 19:40:11.486246 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:11.486218    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w5-e9de60-85ffddb78-6zw8w"]
Oct 30 19:40:11.486371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:11.486336    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w5-e9de60-85ffddb78-6zw8w"
Oct 30 19:40:11.596716 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:11.592700    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7n6js\" (UniqueName: \"kubernetes.io/projected/fb7394c6-8f11-4b08-954e-778579e3ec54-kube-api-access-7n6js\") pod \"payload-processing-e2e-shared-b-w5-e9de60-85ffddb78-6zw8w\" (UID: \"fb7394c6-8f11-4b08-954e-778579e3ec54\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w5-e9de60-85ffddb78-6zw8w"
Oct 30 19:40:11.596716 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:11.593185    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-e9de60-75896ff6bc-vw4jn"
Oct 30 19:40:11.596716 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:11.593839    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb7394c6-8f11-4b08-954e-778579e3ec54-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w5-e9de60-85ffddb78-6zw8w\" (UID: \"fb7394c6-8f11-4b08-954e-778579e3ec54\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w5-e9de60-85ffddb78-6zw8w"
Oct 30 19:40:11.696637 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:11.696446    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb7394c6-8f11-4b08-954e-778579e3ec54-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w5-e9de60-85ffddb78-6zw8w\" (UID: \"fb7394c6-8f11-4b08-954e-778579e3ec54\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w5-e9de60-85ffddb78-6zw8w"
Oct 30 19:40:11.697495 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:11.697457    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb7394c6-8f11-4b08-954e-778579e3ec54-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w5-e9de60-85ffddb78-6zw8w\" (UID: \"fb7394c6-8f11-4b08-954e-778579e3ec54\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w5-e9de60-85ffddb78-6zw8w"
Oct 30 19:40:11.697661 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:11.697642    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7n6js\" (UniqueName: \"kubernetes.io/projected/fb7394c6-8f11-4b08-954e-778579e3ec54-kube-api-access-7n6js\") pod \"payload-processing-e2e-shared-b-w5-e9de60-85ffddb78-6zw8w\" (UID: \"fb7394c6-8f11-4b08-954e-778579e3ec54\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w5-e9de60-85ffddb78-6zw8w"
Oct 30 19:40:11.719382 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:11.719352    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7n6js\" (UniqueName: \"kubernetes.io/projected/fb7394c6-8f11-4b08-954e-778579e3ec54-kube-api-access-7n6js\") pod \"payload-processing-e2e-shared-b-w5-e9de60-85ffddb78-6zw8w\" (UID: \"fb7394c6-8f11-4b08-954e-778579e3ec54\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w5-e9de60-85ffddb78-6zw8w"
Oct 30 19:40:11.878935 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:11.867174    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w5-e9de60-85ffddb78-6zw8w"
Oct 30 19:40:12.005288 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:12.005104    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w5-e9de60-75896ff6bc-vw4jn"]
Oct 30 19:40:12.249412 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:12.247456    5907 patch_prober.go:29] interesting pod/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.159:15021/healthz/ready\": read tcp 10.134.0.2:34976->10.134.0.159:15021: read: connection reset by peer" start-of-body=
Oct 30 19:40:12.249412 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:12.247516    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp" podUID="6662d5b3-a2a2-4187-ae74-afaeaa1b82e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.159:15021/healthz/ready\": read tcp 10.134.0.2:34976->10.134.0.159:15021: read: connection reset by peer"
Oct 30 19:40:12.293109 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:12.293076    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d7ee7826-fc21-4470-9ae7-1db1fb50c50d" path="/var/lib/kubelet/pods/d7ee7826-fc21-4470-9ae7-1db1fb50c50d/volumes"
Oct 30 19:40:12.328637 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:40:12.328606    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb7394c6_8f11_4b08_954e_778579e3ec54.slice/crio-a37fdd0faccb87517827609acf9706473fb8c72f9849b4be344bcc946e621d46 WatchSource:0}: Error finding container a37fdd0faccb87517827609acf9706473fb8c72f9849b4be344bcc946e621d46: Status 404 returned error can't find the container with id a37fdd0faccb87517827609acf9706473fb8c72f9849b4be344bcc946e621d46
Oct 30 19:40:12.330555 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:12.330534    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w5-e9de60-85ffddb78-6zw8w"]
Oct 30 19:40:12.646922 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:12.646886    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-e9de60-75896ff6bc-vw4jn" event={"ID":"efc8d0b8-a0b1-4c98-be74-87a9fcde2b96","Type":"ContainerStarted","Data":"d0dd28f211716e79c46f11d6cb847d89ab63379311a64285d6062347a66b4ec5"}
Oct 30 19:40:12.647147 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:12.646951    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-e9de60-75896ff6bc-vw4jn" event={"ID":"efc8d0b8-a0b1-4c98-be74-87a9fcde2b96","Type":"ContainerStarted","Data":"95813788716eb1465b0b1f4d715befba439c88d264d8991439d73889d9ca9b0c"}
Oct 30 19:40:12.647147 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:12.646986    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-e9de60-75896ff6bc-vw4jn"
Oct 30 19:40:12.648958 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:12.648933    5907 generic.go:356] "Generic (PLEG): container finished" podID="6662d5b3-a2a2-4187-ae74-afaeaa1b82e4" containerID="148ea3c66d89c893e746c8d7bda4d25f4ca0c29824aaa93c6389a86d90bca3be" exitCode=0
Oct 30 19:40:12.649110 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:12.649009    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp" event={"ID":"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4","Type":"ContainerDied","Data":"148ea3c66d89c893e746c8d7bda4d25f4ca0c29824aaa93c6389a86d90bca3be"}
Oct 30 19:40:12.649439 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:12.649421    5907 scope.go:122] "RemoveContainer" containerID="148ea3c66d89c893e746c8d7bda4d25f4ca0c29824aaa93c6389a86d90bca3be"
Oct 30 19:40:12.651302 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:12.651281    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w5-e9de60-85ffddb78-6zw8w" event={"ID":"fb7394c6-8f11-4b08-954e-778579e3ec54","Type":"ContainerStarted","Data":"ba86bbda5a0ab25ce6f597ad03f09b6e40162e442ed4c75316e4dfbdc3c5d817"}
Oct 30 19:40:12.651408 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:12.651322    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w5-e9de60-85ffddb78-6zw8w" event={"ID":"fb7394c6-8f11-4b08-954e-778579e3ec54","Type":"ContainerStarted","Data":"a37fdd0faccb87517827609acf9706473fb8c72f9849b4be344bcc946e621d46"}
Oct 30 19:40:12.651408 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:12.651367    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:40:12.651498 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:12.651382    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w5-e9de60-85ffddb78-6zw8w"
Oct 30 19:40:12.651537 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:12.651459    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:40:12.651576 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:12.651549    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:40:12.697010 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:12.696951    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-e9de60-75896ff6bc-vw4jn" podStartSLOduration=2.696933736 podStartE2EDuration="2.696933736s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:40:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:40:12.693757513 +0000 UTC m=+1124.536689452" watchObservedRunningTime="2026-10-30 19:40:12.696933736 +0000 UTC m=+1124.539865675"
Oct 30 19:40:12.780782 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:12.780719    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w5-e9de60-85ffddb78-6zw8w" podStartSLOduration=1.780696634 podStartE2EDuration="1.780696634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:40:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:40:12.779109481 +0000 UTC m=+1124.622041420" watchObservedRunningTime="2026-10-30 19:40:12.780696634 +0000 UTC m=+1124.623628578"
Oct 30 19:40:13.700542 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:13.700504    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp" event={"ID":"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4","Type":"ContainerStarted","Data":"269c03112689e83620338c8c4dcba3c0e6bdad7380c401ad8aa58735fb6b3299"}
Oct 30 19:40:14.254335 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:14.254300    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:40:14.254597 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:14.254530    5907 patch_prober.go:29] interesting pod/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused" start-of-body=
Oct 30 19:40:14.254597 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:14.254578    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp" podUID="6662d5b3-a2a2-4187-ae74-afaeaa1b82e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused"
Oct 30 19:40:15.255319 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:15.255280    5907 patch_prober.go:29] interesting pod/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused" start-of-body=
Oct 30 19:40:15.256015 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:15.255980    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp" podUID="6662d5b3-a2a2-4187-ae74-afaeaa1b82e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused"
Oct 30 19:40:16.254671 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:16.254635    5907 patch_prober.go:29] interesting pod/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused" start-of-body=
Oct 30 19:40:16.254871 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:16.254705    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp" podUID="6662d5b3-a2a2-4187-ae74-afaeaa1b82e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused"
Oct 30 19:40:16.742675 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:16.742636    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"]
Oct 30 19:40:16.751523 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:16.751496    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:40:16.777341 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:16.774592    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"]
Oct 30 19:40:16.777341 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:16.776607    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-34bdc0-openshift-default-dockercfg-fh8wm\""
Oct 30 19:40:16.869891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:16.863915    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvsz2\" (UniqueName: \"kubernetes.io/projected/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-kube-api-access-fvsz2\") pod \"e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") " pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:40:16.869891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:16.864021    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-istio-podinfo\") pod \"e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") " pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:40:16.869891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:16.864134    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-workload-socket\") pod \"e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") " pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:40:16.869891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:16.864202    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-istio-token\") pod \"e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") " pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:40:16.869891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:16.864266    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-istiod-ca-cert\") pod \"e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") " pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:40:16.869891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:16.864366    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-credential-socket\") pod \"e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") " pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:40:16.869891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:16.864416    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-workload-certs\") pod \"e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") " pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:40:16.869891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:16.864465    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-istio-data\") pod \"e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") " pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:40:16.869891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:16.864505    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-istio-envoy\") pod \"e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") " pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:40:16.975892 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:16.972096    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-istio-podinfo\") pod \"e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") " pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:40:16.975892 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:16.972230    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-workload-socket\") pod \"e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") " pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:40:16.975892 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:16.972298    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-istio-token\") pod \"e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") " pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:40:16.975892 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:16.972361    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-istiod-ca-cert\") pod \"e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") " pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:40:16.975892 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:16.972456    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-credential-socket\") pod \"e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") " pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:40:16.975892 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:16.972505    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-workload-certs\") pod \"e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") " pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:40:16.975892 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:16.972548    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-istio-data\") pod \"e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") " pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:40:16.975892 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:16.972585    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-istio-envoy\") pod \"e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") " pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:40:16.975892 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:16.972784    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvsz2\" (UniqueName: \"kubernetes.io/projected/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-kube-api-access-fvsz2\") pod \"e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") " pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:40:17.021715 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:16.988606    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-workload-certs\") pod \"e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") " pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:40:17.021715 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:16.989236    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-workload-socket\") pod \"e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") " pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:40:17.021715 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:16.990212    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-istiod-ca-cert\") pod \"e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") " pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:40:17.043351 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:17.043316    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-credential-socket\") pod \"e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") " pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:40:17.049927 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:17.049177    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-istio-data\") pod \"e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") " pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:40:17.168909 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:17.151590    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-istio-envoy\") pod \"e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") " pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:40:17.176786 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:17.176751    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-istio-token\") pod \"e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") " pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:40:17.213925 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:17.213892    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvsz2\" (UniqueName: \"kubernetes.io/projected/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-kube-api-access-fvsz2\") pod \"e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") " pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:40:17.215295 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:17.215269    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-istio-podinfo\") pod \"e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") " pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:40:17.258403 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:17.258368    5907 patch_prober.go:29] interesting pod/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused" start-of-body=
Oct 30 19:40:17.258759 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:17.258714    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp" podUID="6662d5b3-a2a2-4187-ae74-afaeaa1b82e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused"
Oct 30 19:40:17.396349 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:17.395383    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:40:17.623487 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:40:17.623418    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:40:17.946533 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:40:17.946501    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76a6fce4_3ba8_434a_b126_8a1a8465b1bf.slice/crio-cf0ac265c923114d51c593ba27671f6c0071a1b96c506ecd6849b0c276b3dd4c WatchSource:0}: Error finding container cf0ac265c923114d51c593ba27671f6c0071a1b96c506ecd6849b0c276b3dd4c: Status 404 returned error can't find the container with id cf0ac265c923114d51c593ba27671f6c0071a1b96c506ecd6849b0c276b3dd4c
Oct 30 19:40:17.947052 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:17.947027    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"]
Oct 30 19:40:17.949562 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:17.949515    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:40:17.949660 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:17.949621    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:40:17.949728 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:17.949674    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:40:18.254581 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:18.254502    5907 patch_prober.go:29] interesting pod/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused" start-of-body=
Oct 30 19:40:18.254581 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:18.254555    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp" podUID="6662d5b3-a2a2-4187-ae74-afaeaa1b82e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused"
Oct 30 19:40:18.847391 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:18.847349    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6" event={"ID":"76a6fce4-3ba8-434a-b126-8a1a8465b1bf","Type":"ContainerStarted","Data":"00f49ab6f04e5d256b9f1fd0b6d2d75b2448c8aab2a7c1828d2e8c119788b7fb"}
Oct 30 19:40:18.847675 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:18.847655    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6" event={"ID":"76a6fce4-3ba8-434a-b126-8a1a8465b1bf","Type":"ContainerStarted","Data":"cf0ac265c923114d51c593ba27671f6c0071a1b96c506ecd6849b0c276b3dd4c"}
Oct 30 19:40:18.950132 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:18.950076    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6" podStartSLOduration=2.950058598 podStartE2EDuration="2.950058598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:40:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:40:18.932391265 +0000 UTC m=+1130.775323205" watchObservedRunningTime="2026-10-30 19:40:18.950058598 +0000 UTC m=+1130.792990536"
Oct 30 19:40:19.254340 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:19.254305    5907 patch_prober.go:29] interesting pod/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused" start-of-body=
Oct 30 19:40:19.254526 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:19.254368    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp" podUID="6662d5b3-a2a2-4187-ae74-afaeaa1b82e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused"
Oct 30 19:40:19.396054 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:19.396019    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:40:19.402351 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:19.402323    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:40:19.858574 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:19.858538    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:40:19.859511 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:19.859489    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:40:20.258237 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:20.258205    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:40:20.258762 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:20.258741    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:40:20.259597 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:20.259568    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:40:21.747922 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:21.747890    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-eb1d94ec-gnqqd"]
Oct 30 19:40:21.753351 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:21.753331    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-eb1d94ec-gnqqd"
Oct 30 19:40:21.766082 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:21.766059    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a472ff69-84d7-4e3a-ac85-317f6f4c95aa-maas-api-xks-ca\") pod \"maas-api-revoke-sub-eb1d94ec-gnqqd\" (UID: \"a472ff69-84d7-4e3a-ac85-317f6f4c95aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-eb1d94ec-gnqqd"
Oct 30 19:40:21.766208 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:21.766101    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6km6\" (UniqueName: \"kubernetes.io/projected/a472ff69-84d7-4e3a-ac85-317f6f4c95aa-kube-api-access-l6km6\") pod \"maas-api-revoke-sub-eb1d94ec-gnqqd\" (UID: \"a472ff69-84d7-4e3a-ac85-317f6f4c95aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-eb1d94ec-gnqqd"
Oct 30 19:40:21.766258 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:21.766226    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a472ff69-84d7-4e3a-ac85-317f6f4c95aa-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-eb1d94ec-gnqqd\" (UID: \"a472ff69-84d7-4e3a-ac85-317f6f4c95aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-eb1d94ec-gnqqd"
Oct 30 19:40:21.773648 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:21.773621    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-eb1d94ec-gnqqd"]
Oct 30 19:40:21.780502 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:21.780473    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:40:21.780618 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:21.780579    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:40:21.867500 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:21.867466    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a472ff69-84d7-4e3a-ac85-317f6f4c95aa-maas-api-xks-ca\") pod \"maas-api-revoke-sub-eb1d94ec-gnqqd\" (UID: \"a472ff69-84d7-4e3a-ac85-317f6f4c95aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-eb1d94ec-gnqqd"
Oct 30 19:40:21.867669 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:21.867528    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6km6\" (UniqueName: \"kubernetes.io/projected/a472ff69-84d7-4e3a-ac85-317f6f4c95aa-kube-api-access-l6km6\") pod \"maas-api-revoke-sub-eb1d94ec-gnqqd\" (UID: \"a472ff69-84d7-4e3a-ac85-317f6f4c95aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-eb1d94ec-gnqqd"
Oct 30 19:40:21.867669 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:21.867617    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a472ff69-84d7-4e3a-ac85-317f6f4c95aa-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-eb1d94ec-gnqqd\" (UID: \"a472ff69-84d7-4e3a-ac85-317f6f4c95aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-eb1d94ec-gnqqd"
Oct 30 19:40:21.868326 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:21.868304    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a472ff69-84d7-4e3a-ac85-317f6f4c95aa-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-eb1d94ec-gnqqd\" (UID: \"a472ff69-84d7-4e3a-ac85-317f6f4c95aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-eb1d94ec-gnqqd"
Oct 30 19:40:21.870205 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:21.870179    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a472ff69-84d7-4e3a-ac85-317f6f4c95aa-maas-api-xks-ca\") pod \"maas-api-revoke-sub-eb1d94ec-gnqqd\" (UID: \"a472ff69-84d7-4e3a-ac85-317f6f4c95aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-eb1d94ec-gnqqd"
Oct 30 19:40:21.888060 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:21.888033    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6km6\" (UniqueName: \"kubernetes.io/projected/a472ff69-84d7-4e3a-ac85-317f6f4c95aa-kube-api-access-l6km6\") pod \"maas-api-revoke-sub-eb1d94ec-gnqqd\" (UID: \"a472ff69-84d7-4e3a-ac85-317f6f4c95aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-eb1d94ec-gnqqd"
Oct 30 19:40:22.070465 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:22.070137    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-eb1d94ec-gnqqd"
Oct 30 19:40:22.483344 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:22.483316    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-eb1d94ec-gnqqd"]
Oct 30 19:40:22.890647 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:22.890616    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-eb1d94ec-gnqqd" event={"ID":"a472ff69-84d7-4e3a-ac85-317f6f4c95aa","Type":"ContainerStarted","Data":"251ada4c994b5ef5c5d030e27994bae130dced9bfbc9c9e8cd0facbc6a68e297"}
Oct 30 19:40:23.104843 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:23.104809    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-34bdc0-6c45d6ff5-dxrqw"]
Oct 30 19:40:23.110003 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:23.109976    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-34bdc0-6c45d6ff5-dxrqw"
Oct 30 19:40:23.133534 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:23.133464    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-34bdc0-56c759c9d-dch72"]
Oct 30 19:40:23.138502 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:23.138456    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-34bdc0\""
Oct 30 19:40:23.140155 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:23.140130    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-34bdc0-56c759c9d-dch72"
Oct 30 19:40:23.140735 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:23.140680    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-34bdc0-6c45d6ff5-dxrqw"]
Oct 30 19:40:23.163004 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:23.162966    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-34bdc0-56c759c9d-dch72"]
Oct 30 19:40:23.166960 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:23.166502    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-34bdc0-dockercfg-fjvjh\""
Oct 30 19:40:23.293887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:23.271329    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee095b66-4a1d-491c-9e6d-3a189a80830a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-34bdc0-56c759c9d-dch72\" (UID: \"ee095b66-4a1d-491c-9e6d-3a189a80830a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-34bdc0-56c759c9d-dch72"
Oct 30 19:40:23.293887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:23.271432    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9n26x\" (UniqueName: \"kubernetes.io/projected/6b44cab3-caa0-42ac-a40e-6f8ff8008c61-kube-api-access-9n26x\") pod \"payload-pre-processing-e2e-shared-a-w3-34bdc0-6c45d6ff5-dxrqw\" (UID: \"6b44cab3-caa0-42ac-a40e-6f8ff8008c61\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-34bdc0-6c45d6ff5-dxrqw"
Oct 30 19:40:23.293887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:23.271757    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vxd4\" (UniqueName: \"kubernetes.io/projected/ee095b66-4a1d-491c-9e6d-3a189a80830a-kube-api-access-7vxd4\") pod \"payload-processing-e2e-shared-a-w3-34bdc0-56c759c9d-dch72\" (UID: \"ee095b66-4a1d-491c-9e6d-3a189a80830a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-34bdc0-56c759c9d-dch72"
Oct 30 19:40:23.293887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:23.271873    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b44cab3-caa0-42ac-a40e-6f8ff8008c61-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-34bdc0-6c45d6ff5-dxrqw\" (UID: \"6b44cab3-caa0-42ac-a40e-6f8ff8008c61\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-34bdc0-6c45d6ff5-dxrqw"
Oct 30 19:40:23.375883 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:23.375812    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee095b66-4a1d-491c-9e6d-3a189a80830a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-34bdc0-56c759c9d-dch72\" (UID: \"ee095b66-4a1d-491c-9e6d-3a189a80830a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-34bdc0-56c759c9d-dch72"
Oct 30 19:40:23.376081 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:23.375935    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9n26x\" (UniqueName: \"kubernetes.io/projected/6b44cab3-caa0-42ac-a40e-6f8ff8008c61-kube-api-access-9n26x\") pod \"payload-pre-processing-e2e-shared-a-w3-34bdc0-6c45d6ff5-dxrqw\" (UID: \"6b44cab3-caa0-42ac-a40e-6f8ff8008c61\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-34bdc0-6c45d6ff5-dxrqw"
Oct 30 19:40:23.376943 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:23.376214    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vxd4\" (UniqueName: \"kubernetes.io/projected/ee095b66-4a1d-491c-9e6d-3a189a80830a-kube-api-access-7vxd4\") pod \"payload-processing-e2e-shared-a-w3-34bdc0-56c759c9d-dch72\" (UID: \"ee095b66-4a1d-491c-9e6d-3a189a80830a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-34bdc0-56c759c9d-dch72"
Oct 30 19:40:23.376943 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:23.376305    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b44cab3-caa0-42ac-a40e-6f8ff8008c61-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-34bdc0-6c45d6ff5-dxrqw\" (UID: \"6b44cab3-caa0-42ac-a40e-6f8ff8008c61\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-34bdc0-6c45d6ff5-dxrqw"
Oct 30 19:40:23.382882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:23.377432    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b44cab3-caa0-42ac-a40e-6f8ff8008c61-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-34bdc0-6c45d6ff5-dxrqw\" (UID: \"6b44cab3-caa0-42ac-a40e-6f8ff8008c61\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-34bdc0-6c45d6ff5-dxrqw"
Oct 30 19:40:23.416051 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:23.401908    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee095b66-4a1d-491c-9e6d-3a189a80830a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-34bdc0-56c759c9d-dch72\" (UID: \"ee095b66-4a1d-491c-9e6d-3a189a80830a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-34bdc0-56c759c9d-dch72"
Oct 30 19:40:23.507492 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:23.506189    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9n26x\" (UniqueName: \"kubernetes.io/projected/6b44cab3-caa0-42ac-a40e-6f8ff8008c61-kube-api-access-9n26x\") pod \"payload-pre-processing-e2e-shared-a-w3-34bdc0-6c45d6ff5-dxrqw\" (UID: \"6b44cab3-caa0-42ac-a40e-6f8ff8008c61\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-34bdc0-6c45d6ff5-dxrqw"
Oct 30 19:40:23.509231 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:23.509201    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vxd4\" (UniqueName: \"kubernetes.io/projected/ee095b66-4a1d-491c-9e6d-3a189a80830a-kube-api-access-7vxd4\") pod \"payload-processing-e2e-shared-a-w3-34bdc0-56c759c9d-dch72\" (UID: \"ee095b66-4a1d-491c-9e6d-3a189a80830a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-34bdc0-56c759c9d-dch72"
Oct 30 19:40:23.729460 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:23.729143    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-34bdc0-6c45d6ff5-dxrqw"
Oct 30 19:40:23.730834 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:23.730800    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-e9de60-75896ff6bc-vw4jn"
Oct 30 19:40:23.734471 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:23.732755    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w5-e9de60-85ffddb78-6zw8w"
Oct 30 19:40:23.769565 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:23.769267    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-34bdc0-56c759c9d-dch72"
Oct 30 19:40:23.913434 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:23.913399    5907 generic.go:356] "Generic (PLEG): container finished" podID="a472ff69-84d7-4e3a-ac85-317f6f4c95aa" containerID="fd88a915d82e01878f1b18fc95a7a462441d0f7674bc43dbbf1cc7940abd5b61" exitCode=0
Oct 30 19:40:23.913890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:23.913498    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-eb1d94ec-gnqqd" event={"ID":"a472ff69-84d7-4e3a-ac85-317f6f4c95aa","Type":"ContainerDied","Data":"fd88a915d82e01878f1b18fc95a7a462441d0f7674bc43dbbf1cc7940abd5b61"}
Oct 30 19:40:24.152021 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:24.151982    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-34bdc0-6c45d6ff5-dxrqw"]
Oct 30 19:40:24.178271 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:24.176316    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-34bdc0-56c759c9d-dch72"]
Oct 30 19:40:24.926143 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:24.926105    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-34bdc0-56c759c9d-dch72" event={"ID":"ee095b66-4a1d-491c-9e6d-3a189a80830a","Type":"ContainerStarted","Data":"fd8c7f4266825f99110ff0d3138c949ae8f053570c5a4614e57d4aceda65b483"}
Oct 30 19:40:24.926601 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:24.926170    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-34bdc0-56c759c9d-dch72" event={"ID":"ee095b66-4a1d-491c-9e6d-3a189a80830a","Type":"ContainerStarted","Data":"5acbf630e17c651986db789205d98cfbf1bd2232d3bae7bf54f77e1956d0ee1b"}
Oct 30 19:40:24.926601 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:24.926200    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-34bdc0-56c759c9d-dch72"
Oct 30 19:40:24.928129 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:24.928099    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-34bdc0-6c45d6ff5-dxrqw" event={"ID":"6b44cab3-caa0-42ac-a40e-6f8ff8008c61","Type":"ContainerStarted","Data":"81a529764aac4a94621e2550564953070b87e90f5151f11fa060a91a7e95fc78"}
Oct 30 19:40:24.928258 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:24.928150    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-34bdc0-6c45d6ff5-dxrqw" event={"ID":"6b44cab3-caa0-42ac-a40e-6f8ff8008c61","Type":"ContainerStarted","Data":"d79282aac46abee980fdf5c7101495bce53f2ff9a57d80dd7276d283ca901fc5"}
Oct 30 19:40:24.971827 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:24.971777    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-34bdc0-56c759c9d-dch72" podStartSLOduration=1.971764769 podStartE2EDuration="1.971764769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:40:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:40:24.966085615 +0000 UTC m=+1136.809017554" watchObservedRunningTime="2026-10-30 19:40:24.971764769 +0000 UTC m=+1136.814696707"
Oct 30 19:40:25.018511 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:25.018464    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-34bdc0-6c45d6ff5-dxrqw" podStartSLOduration=2.018450556 podStartE2EDuration="2.018450556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:40:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:40:25.006261964 +0000 UTC m=+1136.849193903" watchObservedRunningTime="2026-10-30 19:40:25.018450556 +0000 UTC m=+1136.861382494"
Oct 30 19:40:25.116788 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:25.116759    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-eb1d94ec-gnqqd"
Oct 30 19:40:25.289405 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:25.289315    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a472ff69-84d7-4e3a-ac85-317f6f4c95aa-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a472ff69-84d7-4e3a-ac85-317f6f4c95aa-maas-api-xks-ca\") pod \"a472ff69-84d7-4e3a-ac85-317f6f4c95aa\" (UID: \"a472ff69-84d7-4e3a-ac85-317f6f4c95aa\") "
Oct 30 19:40:25.289599 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:25.289515    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a472ff69-84d7-4e3a-ac85-317f6f4c95aa-kube-api-access-l6km6\" (UniqueName: \"kubernetes.io/projected/a472ff69-84d7-4e3a-ac85-317f6f4c95aa-kube-api-access-l6km6\") pod \"a472ff69-84d7-4e3a-ac85-317f6f4c95aa\" (UID: \"a472ff69-84d7-4e3a-ac85-317f6f4c95aa\") "
Oct 30 19:40:25.289667 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:25.289642    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a472ff69-84d7-4e3a-ac85-317f6f4c95aa-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a472ff69-84d7-4e3a-ac85-317f6f4c95aa-maas-api-openshift-ca\") pod \"a472ff69-84d7-4e3a-ac85-317f6f4c95aa\" (UID: \"a472ff69-84d7-4e3a-ac85-317f6f4c95aa\") "
Oct 30 19:40:25.291036 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:25.291008    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a472ff69-84d7-4e3a-ac85-317f6f4c95aa-maas-api-openshift-ca" pod "a472ff69-84d7-4e3a-ac85-317f6f4c95aa" (UID: "a472ff69-84d7-4e3a-ac85-317f6f4c95aa"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:40:25.297095 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:25.297034    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a472ff69-84d7-4e3a-ac85-317f6f4c95aa-maas-api-xks-ca" pod "a472ff69-84d7-4e3a-ac85-317f6f4c95aa" (UID: "a472ff69-84d7-4e3a-ac85-317f6f4c95aa"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:40:25.297990 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:25.297961    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a472ff69-84d7-4e3a-ac85-317f6f4c95aa-kube-api-access-l6km6" pod "a472ff69-84d7-4e3a-ac85-317f6f4c95aa" (UID: "a472ff69-84d7-4e3a-ac85-317f6f4c95aa"). InnerVolumeSpecName "kube-api-access-l6km6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:40:25.392917 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:25.392881    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a472ff69-84d7-4e3a-ac85-317f6f4c95aa-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:40:25.392917 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:25.392918    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6km6\" (UniqueName: \"kubernetes.io/projected/a472ff69-84d7-4e3a-ac85-317f6f4c95aa-kube-api-access-l6km6\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:40:25.393119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:25.392934    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a472ff69-84d7-4e3a-ac85-317f6f4c95aa-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:40:25.960885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:25.950343    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-eb1d94ec-gnqqd"
Oct 30 19:40:25.960885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:25.950975    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-eb1d94ec-gnqqd" event={"ID":"a472ff69-84d7-4e3a-ac85-317f6f4c95aa","Type":"ContainerDied","Data":"251ada4c994b5ef5c5d030e27994bae130dced9bfbc9c9e8cd0facbc6a68e297"}
Oct 30 19:40:25.960885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:25.951020    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="251ada4c994b5ef5c5d030e27994bae130dced9bfbc9c9e8cd0facbc6a68e297"
Oct 30 19:40:25.960885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:25.951844    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-34bdc0-6c45d6ff5-dxrqw"
Oct 30 19:40:27.067004 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:27.063913    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6e50b175-l7ggb"]
Oct 30 19:40:27.072827 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:27.072790    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6e50b175-l7ggb"]
Oct 30 19:40:28.291006 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:28.290967    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e1415b6b-73cc-4d05-90c3-00d3df4b3188" path="/var/lib/kubelet/pods/e1415b6b-73cc-4d05-90c3-00d3df4b3188/volumes"
Oct 30 19:40:29.513363 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:29.513333    5907 scope.go:122] "RemoveContainer" containerID="48d8d2e419c68453738064eefb299e31572ff50c4d074e309e60bfd2d43129ff"
Oct 30 19:40:29.525596 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:29.525566    5907 scope.go:122] "RemoveContainer" containerID="55986730e1b950b0060a304170d843ed7e7a9c3e95cd54a0f10a59f113df1649"
Oct 30 19:40:29.538198 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:29.538091    5907 scope.go:122] "RemoveContainer" containerID="8f310e7abe98f21ae923d552c8a1efa4fb92a2582f65ce8be692726341f50579"
Oct 30 19:40:29.550096 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:29.550073    5907 scope.go:122] "RemoveContainer" containerID="3ef07aa89761b08ac6bd7a20aed449b36a88ba5dcd57da4254749d55a3343afb"
Oct 30 19:40:29.560047 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:29.560030    5907 scope.go:122] "RemoveContainer" containerID="1b1c0063983ad87aa52f461c6c974456982e7f4911322ba92b2b7b91d2ac16be"
Oct 30 19:40:29.569780 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:29.569761    5907 scope.go:122] "RemoveContainer" containerID="432a6fddc5664657187e7164d6a05254d4fc9a35e6ec9928d80a510894254087"
Oct 30 19:40:29.579760 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:29.579741    5907 scope.go:122] "RemoveContainer" containerID="5dd8d3cf391195e86379f1e00952a0d12250f9a88d0376ba87b36de439c44870"
Oct 30 19:40:29.589874 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:29.589845    5907 scope.go:122] "RemoveContainer" containerID="4d821bd0714776f533c0ef2a043171eeed181d381462c12e68e76945752d94ee"
Oct 30 19:40:29.599407 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:29.599392    5907 scope.go:122] "RemoveContainer" containerID="b2d9470e5dc990dfa05440af3ae53f9b834a47afe7f770ec4f49385de31e19c7"
Oct 30 19:40:29.608809 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:29.608795    5907 scope.go:122] "RemoveContainer" containerID="1939eda93c46b905411846a2bb139993377326ffb3d5187095d33be9ff178a40"
Oct 30 19:40:29.618658 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:29.618640    5907 scope.go:122] "RemoveContainer" containerID="2f98c98e6b9104629c935ea6b0d73421bdb4a8b4e5518fddefe9d6de9d0a4a0e"
Oct 30 19:40:29.629121 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:29.629096    5907 scope.go:122] "RemoveContainer" containerID="3a534aef16c7cb76926db7453cf2f9722eccf03786e1daf4869bfc1c451308fa"
Oct 30 19:40:35.953888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:35.953830    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-34bdc0-56c759c9d-dch72"
Oct 30 19:40:36.044644 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:36.044609    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-305b2a9e-ssljn"]
Oct 30 19:40:36.050122 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:36.050096    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-305b2a9e-ssljn"]
Oct 30 19:40:36.290545 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:36.290469    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="41c4941d-9ed5-40d1-9f24-3ffb919eebeb" path="/var/lib/kubelet/pods/41c4941d-9ed5-40d1-9f24-3ffb919eebeb/volumes"
Oct 30 19:40:36.962439 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:36.962410    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-34bdc0-6c45d6ff5-dxrqw"
Oct 30 19:40:43.032130 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:43.032095    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-af2965cf-tv9tb"]
Oct 30 19:40:43.045702 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:43.045672    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-af2965cf-tv9tb"]
Oct 30 19:40:44.289957 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:44.289923    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="55c624df-a753-4418-9f4d-1e277205e7da" path="/var/lib/kubelet/pods/55c624df-a753-4418-9f4d-1e277205e7da/volumes"
Oct 30 19:40:47.504812 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:40:47.504776    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:40:51.896738 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:51.896695    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"]
Oct 30 19:40:51.898186 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:51.898158    5907 cpu_manager.go:414] "containerMap: removing container" podUID="a472ff69-84d7-4e3a-ac85-317f6f4c95aa" containerName="revoke-keys"
Oct 30 19:40:51.898186 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:51.898186    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a472ff69-84d7-4e3a-ac85-317f6f4c95aa" containerName="revoke-keys"
Oct 30 19:40:51.898438 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:51.898420    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="a472ff69-84d7-4e3a-ac85-317f6f4c95aa" containerName="revoke-keys"
Oct 30 19:40:51.903585 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:51.903563    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:40:51.933887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:51.933845    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-67953efd-openshift-default-dockercfg-74jjs\""
Oct 30 19:40:51.936064 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:51.936038    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"]
Oct 30 19:40:52.079419 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:52.079372    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0435effb-f3d6-450f-beed-937cec7badb0-istio-envoy\") pod \"e2e-mt-67953efd-openshift-default-f666c5445-7cxn8\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") " pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:40:52.079815 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:52.079788    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0435effb-f3d6-450f-beed-937cec7badb0-istio-token\") pod \"e2e-mt-67953efd-openshift-default-f666c5445-7cxn8\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") " pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:40:52.089896 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:52.088953    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0435effb-f3d6-450f-beed-937cec7badb0-istiod-ca-cert\") pod \"e2e-mt-67953efd-openshift-default-f666c5445-7cxn8\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") " pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:40:52.089896 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:52.089046    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glklx\" (UniqueName: \"kubernetes.io/projected/0435effb-f3d6-450f-beed-937cec7badb0-kube-api-access-glklx\") pod \"e2e-mt-67953efd-openshift-default-f666c5445-7cxn8\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") " pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:40:52.089896 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:52.089148    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0435effb-f3d6-450f-beed-937cec7badb0-istio-data\") pod \"e2e-mt-67953efd-openshift-default-f666c5445-7cxn8\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") " pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:40:52.089896 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:52.089317    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0435effb-f3d6-450f-beed-937cec7badb0-workload-certs\") pod \"e2e-mt-67953efd-openshift-default-f666c5445-7cxn8\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") " pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:40:52.089896 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:52.089352    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0435effb-f3d6-450f-beed-937cec7badb0-istio-podinfo\") pod \"e2e-mt-67953efd-openshift-default-f666c5445-7cxn8\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") " pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:40:52.089896 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:52.089424    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0435effb-f3d6-450f-beed-937cec7badb0-credential-socket\") pod \"e2e-mt-67953efd-openshift-default-f666c5445-7cxn8\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") " pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:40:52.089896 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:52.089643    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0435effb-f3d6-450f-beed-937cec7badb0-workload-socket\") pod \"e2e-mt-67953efd-openshift-default-f666c5445-7cxn8\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") " pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:40:52.203923 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:52.203780    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0435effb-f3d6-450f-beed-937cec7badb0-istio-envoy\") pod \"e2e-mt-67953efd-openshift-default-f666c5445-7cxn8\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") " pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:40:52.219185 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:52.219152    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0435effb-f3d6-450f-beed-937cec7badb0-istio-token\") pod \"e2e-mt-67953efd-openshift-default-f666c5445-7cxn8\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") " pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:40:52.219604 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:52.219582    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0435effb-f3d6-450f-beed-937cec7badb0-istiod-ca-cert\") pod \"e2e-mt-67953efd-openshift-default-f666c5445-7cxn8\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") " pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:40:52.220301 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:52.220281    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glklx\" (UniqueName: \"kubernetes.io/projected/0435effb-f3d6-450f-beed-937cec7badb0-kube-api-access-glklx\") pod \"e2e-mt-67953efd-openshift-default-f666c5445-7cxn8\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") " pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:40:52.238922 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:52.238891    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0435effb-f3d6-450f-beed-937cec7badb0-istio-data\") pod \"e2e-mt-67953efd-openshift-default-f666c5445-7cxn8\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") " pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:40:52.239745 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:52.239718    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0435effb-f3d6-450f-beed-937cec7badb0-workload-certs\") pod \"e2e-mt-67953efd-openshift-default-f666c5445-7cxn8\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") " pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:40:52.240494 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:52.240474    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0435effb-f3d6-450f-beed-937cec7badb0-istio-podinfo\") pod \"e2e-mt-67953efd-openshift-default-f666c5445-7cxn8\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") " pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:40:52.240705 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:52.240688    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0435effb-f3d6-450f-beed-937cec7badb0-workload-certs\") pod \"e2e-mt-67953efd-openshift-default-f666c5445-7cxn8\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") " pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:40:52.240963 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:52.240142    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0435effb-f3d6-450f-beed-937cec7badb0-istio-data\") pod \"e2e-mt-67953efd-openshift-default-f666c5445-7cxn8\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") " pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:40:52.241131 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:52.241113    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0435effb-f3d6-450f-beed-937cec7badb0-credential-socket\") pod \"e2e-mt-67953efd-openshift-default-f666c5445-7cxn8\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") " pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:40:52.242186 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:52.241837    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0435effb-f3d6-450f-beed-937cec7badb0-istiod-ca-cert\") pod \"e2e-mt-67953efd-openshift-default-f666c5445-7cxn8\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") " pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:40:52.251895 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:52.244016    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0435effb-f3d6-450f-beed-937cec7badb0-workload-socket\") pod \"e2e-mt-67953efd-openshift-default-f666c5445-7cxn8\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") " pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:40:52.252667 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:52.252614    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0435effb-f3d6-450f-beed-937cec7badb0-workload-socket\") pod \"e2e-mt-67953efd-openshift-default-f666c5445-7cxn8\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") " pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:40:52.260958 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:52.260924    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0435effb-f3d6-450f-beed-937cec7badb0-credential-socket\") pod \"e2e-mt-67953efd-openshift-default-f666c5445-7cxn8\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") " pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:40:52.325376 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:52.319825    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-glklx\" (UniqueName: \"kubernetes.io/projected/0435effb-f3d6-450f-beed-937cec7badb0-kube-api-access-glklx\") pod \"e2e-mt-67953efd-openshift-default-f666c5445-7cxn8\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") " pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:40:52.325376 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:52.320093    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0435effb-f3d6-450f-beed-937cec7badb0-istio-envoy\") pod \"e2e-mt-67953efd-openshift-default-f666c5445-7cxn8\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") " pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:40:52.325376 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:52.322423    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0435effb-f3d6-450f-beed-937cec7badb0-istio-podinfo\") pod \"e2e-mt-67953efd-openshift-default-f666c5445-7cxn8\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") " pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:40:52.325376 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:52.322443    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0435effb-f3d6-450f-beed-937cec7badb0-istio-token\") pod \"e2e-mt-67953efd-openshift-default-f666c5445-7cxn8\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") " pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:40:52.533332 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:52.533180    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:40:53.034548 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:40:53.034520    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0435effb_f3d6_450f_beed_937cec7badb0.slice/crio-352dcd587e8c2693387ea6c97d091395ae229e2a1f2e919e8c9d641aadb07cd0 WatchSource:0}: Error finding container 352dcd587e8c2693387ea6c97d091395ae229e2a1f2e919e8c9d641aadb07cd0: Status 404 returned error can't find the container with id 352dcd587e8c2693387ea6c97d091395ae229e2a1f2e919e8c9d641aadb07cd0
Oct 30 19:40:53.035530 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:53.035506    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"]
Oct 30 19:40:53.037799 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:53.037765    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:40:53.037926 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:53.037830    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:40:53.037926 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:53.037878    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:40:53.608427 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:53.608378    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8" event={"ID":"0435effb-f3d6-450f-beed-937cec7badb0","Type":"ContainerStarted","Data":"e26d6b861043f3fc8b434d4774de412d9c28fc362a35d3c2135c4569742f2e00"}
Oct 30 19:40:53.608610 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:53.608454    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8" event={"ID":"0435effb-f3d6-450f-beed-937cec7badb0","Type":"ContainerStarted","Data":"352dcd587e8c2693387ea6c97d091395ae229e2a1f2e919e8c9d641aadb07cd0"}
Oct 30 19:40:53.666335 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:53.666277    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8" podStartSLOduration=2.6662568330000003 podStartE2EDuration="2.666256833s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:40:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:40:53.664638167 +0000 UTC m=+1165.507570106" watchObservedRunningTime="2026-10-30 19:40:53.666256833 +0000 UTC m=+1165.509188772"
Oct 30 19:40:54.558882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:54.552951    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:40:54.570042 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:54.569539    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:40:54.625703 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:54.625672    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:40:54.626792 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:54.626770    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:40:55.031283 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:55.031253    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7147304e-gk65b"]
Oct 30 19:40:55.036244 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:55.036218    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7147304e-gk65b"]
Oct 30 19:40:56.290556 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:56.290524    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="407d7b1c-719b-44ed-a746-e2ddbecd1798" path="/var/lib/kubelet/pods/407d7b1c-719b-44ed-a746-e2ddbecd1798/volumes"
Oct 30 19:40:57.839470 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:57.839437    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-sfh2c"]
Oct 30 19:40:57.853115 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:57.853076    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-sfh2c"]
Oct 30 19:40:57.853302 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:57.853199    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-sfh2c"
Oct 30 19:40:57.955910 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:57.946060    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-67953efd-dockercfg-lslns\""
Oct 30 19:40:57.955910 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:57.946766    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-67953efd\""
Oct 30 19:40:57.988900 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:57.988174    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/307b0aff-2386-4219-b34f-5a187231f5a2-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-67953efd-6797bd4db7-sfh2c\" (UID: \"307b0aff-2386-4219-b34f-5a187231f5a2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-sfh2c"
Oct 30 19:40:57.988900 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:57.988267    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhmhx\" (UniqueName: \"kubernetes.io/projected/307b0aff-2386-4219-b34f-5a187231f5a2-kube-api-access-mhmhx\") pod \"payload-pre-processing-e2e-mt-67953efd-6797bd4db7-sfh2c\" (UID: \"307b0aff-2386-4219-b34f-5a187231f5a2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-sfh2c"
Oct 30 19:40:58.098663 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:58.098568    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/307b0aff-2386-4219-b34f-5a187231f5a2-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-67953efd-6797bd4db7-sfh2c\" (UID: \"307b0aff-2386-4219-b34f-5a187231f5a2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-sfh2c"
Oct 30 19:40:58.098960 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:58.098934    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhmhx\" (UniqueName: \"kubernetes.io/projected/307b0aff-2386-4219-b34f-5a187231f5a2-kube-api-access-mhmhx\") pod \"payload-pre-processing-e2e-mt-67953efd-6797bd4db7-sfh2c\" (UID: \"307b0aff-2386-4219-b34f-5a187231f5a2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-sfh2c"
Oct 30 19:40:58.113895 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:58.100495    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/307b0aff-2386-4219-b34f-5a187231f5a2-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-67953efd-6797bd4db7-sfh2c\" (UID: \"307b0aff-2386-4219-b34f-5a187231f5a2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-sfh2c"
Oct 30 19:40:58.240592 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:58.240561    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhmhx\" (UniqueName: \"kubernetes.io/projected/307b0aff-2386-4219-b34f-5a187231f5a2-kube-api-access-mhmhx\") pod \"payload-pre-processing-e2e-mt-67953efd-6797bd4db7-sfh2c\" (UID: \"307b0aff-2386-4219-b34f-5a187231f5a2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-sfh2c"
Oct 30 19:40:58.473779 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:58.473745    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-sfh2c"
Oct 30 19:40:58.789725 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:40:58.789682    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod307b0aff_2386_4219_b34f_5a187231f5a2.slice/crio-2d49259e11c6fb6625b4c39893aaa7f64e573690d15a03681cc2601ce4d79560 WatchSource:0}: Error finding container 2d49259e11c6fb6625b4c39893aaa7f64e573690d15a03681cc2601ce4d79560: Status 404 returned error can't find the container with id 2d49259e11c6fb6625b4c39893aaa7f64e573690d15a03681cc2601ce4d79560
Oct 30 19:40:58.791130 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:58.791106    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-sfh2c"]
Oct 30 19:40:58.983889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:40:58.977243    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:41:00.104896 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:00.091405    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-sfh2c" event={"ID":"307b0aff-2386-4219-b34f-5a187231f5a2","Type":"ContainerStarted","Data":"046a92c6ecd48c344653b0d674d38e089b45ad4a4029ef14bc0253d041875c54"}
Oct 30 19:41:00.104896 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:00.091479    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-sfh2c" event={"ID":"307b0aff-2386-4219-b34f-5a187231f5a2","Type":"ContainerStarted","Data":"2d49259e11c6fb6625b4c39893aaa7f64e573690d15a03681cc2601ce4d79560"}
Oct 30 19:41:00.137983 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:00.125879    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-sfh2c"
Oct 30 19:41:00.208644 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:00.201959    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-sfh2c" podStartSLOduration=3.201924775 podStartE2EDuration="3.201924775s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:40:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:41:00.175205823 +0000 UTC m=+1172.018137764" watchObservedRunningTime="2026-10-30 19:41:00.201924775 +0000 UTC m=+1172.044856715"
Oct 30 19:41:00.451560 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:00.451527    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c6fd28cf-6rjhc"]
Oct 30 19:41:00.515429 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:00.515397    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c6fd28cf-6rjhc"
Oct 30 19:41:00.556113 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:00.556084    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:41:00.573270 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:00.573237    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:41:00.730125 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:00.730017    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c6fd28cf-6rjhc"]
Oct 30 19:41:00.778590 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:00.778556    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6f061b05-d54a-4ea1-a794-08e0c0eafeb2-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-mt-c6fd28cf-6rjhc\" (UID: \"6f061b05-d54a-4ea1-a794-08e0c0eafeb2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c6fd28cf-6rjhc"
Oct 30 19:41:00.779082 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:00.779054    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xd58t\" (UniqueName: \"kubernetes.io/projected/6f061b05-d54a-4ea1-a794-08e0c0eafeb2-kube-api-access-xd58t\") pod \"maas-api-revoke-keys-e2e-mt-c6fd28cf-6rjhc\" (UID: \"6f061b05-d54a-4ea1-a794-08e0c0eafeb2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c6fd28cf-6rjhc"
Oct 30 19:41:00.779322 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:00.779306    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6f061b05-d54a-4ea1-a794-08e0c0eafeb2-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-mt-c6fd28cf-6rjhc\" (UID: \"6f061b05-d54a-4ea1-a794-08e0c0eafeb2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c6fd28cf-6rjhc"
Oct 30 19:41:00.905721 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:00.905685    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6f061b05-d54a-4ea1-a794-08e0c0eafeb2-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-mt-c6fd28cf-6rjhc\" (UID: \"6f061b05-d54a-4ea1-a794-08e0c0eafeb2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c6fd28cf-6rjhc"
Oct 30 19:41:00.914304 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:00.914264    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6f061b05-d54a-4ea1-a794-08e0c0eafeb2-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-mt-c6fd28cf-6rjhc\" (UID: \"6f061b05-d54a-4ea1-a794-08e0c0eafeb2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c6fd28cf-6rjhc"
Oct 30 19:41:00.935890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:00.935813    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd58t\" (UniqueName: \"kubernetes.io/projected/6f061b05-d54a-4ea1-a794-08e0c0eafeb2-kube-api-access-xd58t\") pod \"maas-api-revoke-keys-e2e-mt-c6fd28cf-6rjhc\" (UID: \"6f061b05-d54a-4ea1-a794-08e0c0eafeb2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c6fd28cf-6rjhc"
Oct 30 19:41:00.962009 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:00.948363    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6f061b05-d54a-4ea1-a794-08e0c0eafeb2-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-mt-c6fd28cf-6rjhc\" (UID: \"6f061b05-d54a-4ea1-a794-08e0c0eafeb2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c6fd28cf-6rjhc"
Oct 30 19:41:00.962009 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:00.956548    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6f061b05-d54a-4ea1-a794-08e0c0eafeb2-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-mt-c6fd28cf-6rjhc\" (UID: \"6f061b05-d54a-4ea1-a794-08e0c0eafeb2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c6fd28cf-6rjhc"
Oct 30 19:41:00.995830 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:00.995730    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xd58t\" (UniqueName: \"kubernetes.io/projected/6f061b05-d54a-4ea1-a794-08e0c0eafeb2-kube-api-access-xd58t\") pod \"maas-api-revoke-keys-e2e-mt-c6fd28cf-6rjhc\" (UID: \"6f061b05-d54a-4ea1-a794-08e0c0eafeb2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c6fd28cf-6rjhc"
Oct 30 19:41:01.262938 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:01.260260    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c6fd28cf-6rjhc"
Oct 30 19:41:02.524904 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:02.471167    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 19:41:02.524904 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:02.471558    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:41:02.524904 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:02.472261    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:41:02.553936 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:02.531973    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:41:03.169648 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:41:03.169509    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f061b05_d54a_4ea1_a794_08e0c0eafeb2.slice/crio-38be2d75976f6e6d3c320359d340a4eb37edb233cda5716d5a593db0085fabc4 WatchSource:0}: Error finding container 38be2d75976f6e6d3c320359d340a4eb37edb233cda5716d5a593db0085fabc4: Status 404 returned error can't find the container with id 38be2d75976f6e6d3c320359d340a4eb37edb233cda5716d5a593db0085fabc4
Oct 30 19:41:03.285420 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:03.268683    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c6fd28cf-6rjhc"]
Oct 30 19:41:03.488905 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:03.364139    5907 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/health\": context deadline exceeded"
Oct 30 19:41:03.512419 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:03.478403    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": context deadline exceeded"
Oct 30 19:41:03.599705 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:03.488011    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 19:41:03.599705 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:03.488053    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 19:41:03.655378 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:03.600015    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:41:03.655378 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:03.600091    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:41:03.655378 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:03.600156    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:41:03.950843 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:03.905435    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:41:04.069747 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:03.987135    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:41:04.069747 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.067627    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"]
Oct 30 19:41:04.093176 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.093143    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-s826m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:41:04.093428 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.093353    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m" podUID="86efaf93-7f28-4e6a-9add-c8bdd87c5261" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:41:04.139914 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.137051    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-c2sv6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:41:04.139914 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.137127    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6" podUID="ffa8893a-530b-4232-8e6e-dbe34b30ccda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:41:04.260941 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.253600    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c6fd28cf-6rjhc" event={"ID":"6f061b05-d54a-4ea1-a794-08e0c0eafeb2","Type":"ContainerStarted","Data":"38be2d75976f6e6d3c320359d340a4eb37edb233cda5716d5a593db0085fabc4"}
Oct 30 19:41:04.410842 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.397266    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:41:04.497887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.483439    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-workload-certs\") pod \"e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") " pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:41:04.497887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.483541    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-istio-data\") pod \"e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") " pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:41:04.497887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.483611    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-istiod-ca-cert\") pod \"e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") " pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:41:04.497887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.483670    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dr8b\" (UniqueName: \"kubernetes.io/projected/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-kube-api-access-7dr8b\") pod \"e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") " pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:41:04.497887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.483750    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-workload-socket\") pod \"e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") " pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:41:04.497887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.484030    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-credential-socket\") pod \"e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") " pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:41:04.497887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.484062    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-istio-envoy\") pod \"e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") " pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:41:04.497887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.484153    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-istio-token\") pod \"e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") " pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:41:04.513672 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.513557    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:41:04.616891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.592093    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-istio-podinfo\") pod \"e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") " pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:41:04.845150 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.758214    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"]
Oct 30 19:41:04.845391 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.765782    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 19:41:04.846279 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.766851    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 19:41:04.874888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.785006    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-pdg78 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:41:04.875798 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.791224    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-7e89da-openshift-default-dockercfg-2kxdc\""
Oct 30 19:41:04.876003 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.875678    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78" podUID="bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:41:04.970119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.963711    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-workload-socket\") pod \"e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") " pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:41:04.970119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.964136    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-credential-socket\") pod \"e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") " pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:41:04.970119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.964173    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-istio-envoy\") pod \"e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") " pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:41:04.970119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.964264    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-istio-token\") pod \"e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") " pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:41:04.970119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.964321    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-istio-podinfo\") pod \"e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") " pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:41:04.970119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.964396    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-workload-certs\") pod \"e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") " pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:41:04.970119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.964474    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-istio-data\") pod \"e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") " pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:41:04.970119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.964562    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-istiod-ca-cert\") pod \"e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") " pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:41:04.970119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.964646    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dr8b\" (UniqueName: \"kubernetes.io/projected/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-kube-api-access-7dr8b\") pod \"e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") " pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:41:04.970119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.965369    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-workload-socket\") pod \"e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") " pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:41:05.062471 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.976424    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-workload-certs\") pod \"e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") " pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:41:05.062471 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.977595    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-istio-data\") pod \"e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") " pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:41:05.062471 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.979295    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-istiod-ca-cert\") pod \"e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") " pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:41:05.062471 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:04.993259    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-credential-socket\") pod \"e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") " pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:41:05.062471 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:05.006813    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-istio-envoy\") pod \"e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") " pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:41:05.062471 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:05.027030    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:41:05.063685 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:05.063649    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-istio-podinfo\") pod \"e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") " pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:41:05.179786 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:05.179696    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dr8b\" (UniqueName: \"kubernetes.io/projected/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-kube-api-access-7dr8b\") pod \"e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") " pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:41:05.186449 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:05.186383    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-istio-token\") pod \"e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") " pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:41:05.442380 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:05.442294    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:41:05.584012 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:05.583961    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:41:05.854345 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:05.854223    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:41:05.864138 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:05.863240    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 19:41:05.901500 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:05.901363    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-9tdql container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:41:05.901500 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:05.901425    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql" podUID="8db50859-98fc-49aa-806b-4fbed56b1d00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:41:06.072894 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:06.040288    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:41:06.728183 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:06.728131    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:41:06.915910 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:06.875163    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:41:06.915910 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:06.875698    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 19:41:07.099001 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:07.095303    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:41:07.267887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:07.245276    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:41:07.267887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:07.245464    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:41:07.267887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:07.245581    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:41:07.302799 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:07.302712    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c6fd28cf-6rjhc" event={"ID":"6f061b05-d54a-4ea1-a794-08e0c0eafeb2","Type":"ContainerStarted","Data":"4f0094a43cda37c653f36c66af261a08a4f80ef27bb102b24b984e9a4ab04e96"}
Oct 30 19:41:07.303018 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:07.302800    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:41:07.303018 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:07.302828    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:41:07.303148 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:07.303032    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:41:07.303148 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:07.303124    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:41:07.303256 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:07.303146    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-75645db4-blv48"]
Oct 30 19:41:07.318951 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:07.318919    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-75645db4-blv48"]
Oct 30 19:41:07.334622 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:07.334555    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-fe4ec3-7944b894b6-6qx48"]
Oct 30 19:41:07.344101 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:07.344074    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe4ec3-7944b894b6-6qx48"
Oct 30 19:41:07.352956 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:07.352920    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-54f5f8a0-jd9kz"]
Oct 30 19:41:07.381469 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:07.379044    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-54f5f8a0-jd9kz"]
Oct 30 19:41:07.388107 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:07.387248    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w1-fe4ec3-dockercfg-ddjxk\""
Oct 30 19:41:07.388107 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:07.387760    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w1-fe4ec3\""
Oct 30 19:41:07.401093 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:07.401017    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-fe4ec3-7944b894b6-6qx48"]
Oct 30 19:41:07.408184 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:07.408125    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c6fd28cf-6rjhc" podStartSLOduration=7.408110128 podStartE2EDuration="7.408110128s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:41:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:41:07.300538302 +0000 UTC m=+1179.143470242" watchObservedRunningTime="2026-10-30 19:41:07.408110128 +0000 UTC m=+1179.251042068"
Oct 30 19:41:07.413904 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:07.411586    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"]
Oct 30 19:41:07.533907 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:07.510831    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnlgn\" (UniqueName: \"kubernetes.io/projected/e876e507-2b04-43fd-b788-b10d7d15748a-kube-api-access-tnlgn\") pod \"payload-processing-e2e-shared-a-w1-fe4ec3-7944b894b6-6qx48\" (UID: \"e876e507-2b04-43fd-b788-b10d7d15748a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe4ec3-7944b894b6-6qx48"
Oct 30 19:41:07.533907 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:07.510924    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e876e507-2b04-43fd-b788-b10d7d15748a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-fe4ec3-7944b894b6-6qx48\" (UID: \"e876e507-2b04-43fd-b788-b10d7d15748a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe4ec3-7944b894b6-6qx48"
Oct 30 19:41:07.923197 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:07.675297    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnlgn\" (UniqueName: \"kubernetes.io/projected/e876e507-2b04-43fd-b788-b10d7d15748a-kube-api-access-tnlgn\") pod \"payload-processing-e2e-shared-a-w1-fe4ec3-7944b894b6-6qx48\" (UID: \"e876e507-2b04-43fd-b788-b10d7d15748a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe4ec3-7944b894b6-6qx48"
Oct 30 19:41:07.923197 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:07.675372    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e876e507-2b04-43fd-b788-b10d7d15748a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-fe4ec3-7944b894b6-6qx48\" (UID: \"e876e507-2b04-43fd-b788-b10d7d15748a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe4ec3-7944b894b6-6qx48"
Oct 30 19:41:07.923197 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:07.676820    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e876e507-2b04-43fd-b788-b10d7d15748a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-fe4ec3-7944b894b6-6qx48\" (UID: \"e876e507-2b04-43fd-b788-b10d7d15748a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe4ec3-7944b894b6-6qx48"
Oct 30 19:41:08.065886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:08.065398    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnlgn\" (UniqueName: \"kubernetes.io/projected/e876e507-2b04-43fd-b788-b10d7d15748a-kube-api-access-tnlgn\") pod \"payload-processing-e2e-shared-a-w1-fe4ec3-7944b894b6-6qx48\" (UID: \"e876e507-2b04-43fd-b788-b10d7d15748a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe4ec3-7944b894b6-6qx48"
Oct 30 19:41:08.273628 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:08.273515    5907 generic.go:356] "Generic (PLEG): container finished" podID="6f061b05-d54a-4ea1-a794-08e0c0eafeb2" containerID="4f0094a43cda37c653f36c66af261a08a4f80ef27bb102b24b984e9a4ab04e96" exitCode=0
Oct 30 19:41:08.273936 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:08.273911    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c6fd28cf-6rjhc" event={"ID":"6f061b05-d54a-4ea1-a794-08e0c0eafeb2","Type":"ContainerDied","Data":"4f0094a43cda37c653f36c66af261a08a4f80ef27bb102b24b984e9a4ab04e96"}
Oct 30 19:41:08.277273 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:08.277247    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp" event={"ID":"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf","Type":"ContainerStarted","Data":"e3a35def6621ba550ad37eeb67feec3899966bcf15d06ef4badc081f7d5849eb"}
Oct 30 19:41:08.277403 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:08.277295    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp" event={"ID":"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf","Type":"ContainerStarted","Data":"7f5eb2bd78a58c7aae9a03ba8b36c0509706c243c18268ba3fa1c396e7cd634d"}
Oct 30 19:41:08.294375 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:08.294343    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3f07cf47-62f3-425e-999a-fd496b530952" path="/var/lib/kubelet/pods/3f07cf47-62f3-425e-999a-fd496b530952/volumes"
Oct 30 19:41:08.295351 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:08.295071    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cd39978b-2e78-405e-b806-a158b0c18c49" path="/var/lib/kubelet/pods/cd39978b-2e78-405e-b806-a158b0c18c49/volumes"
Oct 30 19:41:08.365044 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:08.364680    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe4ec3-7944b894b6-6qx48"
Oct 30 19:41:08.405435 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:08.401565    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp" podStartSLOduration=5.401545733 podStartE2EDuration="5.401545733s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:41:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:41:08.396022731 +0000 UTC m=+1180.238954662" watchObservedRunningTime="2026-10-30 19:41:08.401545733 +0000 UTC m=+1180.244477675"
Oct 30 19:41:08.470891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:08.470023    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:41:08.473526 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:08.473210    5907 patch_prober.go:29] interesting pod/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.172:15021/healthz/ready\": dial tcp 10.134.0.172:15021: connect: connection refused" start-of-body=
Oct 30 19:41:08.473526 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:08.473276    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp" podUID="29fbb7be-9068-40ec-a0d4-9fa0ed0469bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.172:15021/healthz/ready\": dial tcp 10.134.0.172:15021: connect: connection refused"
Oct 30 19:41:09.281903 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:09.281848    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-fe4ec3-7944b894b6-6qx48"]
Oct 30 19:41:09.300174 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:09.299057    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe4ec3-7944b894b6-6qx48" event={"ID":"e876e507-2b04-43fd-b788-b10d7d15748a","Type":"ContainerStarted","Data":"a4fa0df8c8fd0059ae3d0d231e0e0234919a473b07b60609b72cc288fd711b82"}
Oct 30 19:41:09.493316 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:09.481418    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:41:09.980188 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:09.980157    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c6fd28cf-6rjhc"
Oct 30 19:41:10.069933 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:10.065936    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6243ccc5-gjbng"]
Oct 30 19:41:10.069933 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:10.067707    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6243ccc5-gjbng"]
Oct 30 19:41:10.141444 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:10.141404    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6f061b05-d54a-4ea1-a794-08e0c0eafeb2-kube-api-access-xd58t\" (UniqueName: \"kubernetes.io/projected/6f061b05-d54a-4ea1-a794-08e0c0eafeb2-kube-api-access-xd58t\") pod \"6f061b05-d54a-4ea1-a794-08e0c0eafeb2\" (UID: \"6f061b05-d54a-4ea1-a794-08e0c0eafeb2\") "
Oct 30 19:41:10.141649 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:10.141481    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6f061b05-d54a-4ea1-a794-08e0c0eafeb2-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6f061b05-d54a-4ea1-a794-08e0c0eafeb2-maas-api-openshift-ca\") pod \"6f061b05-d54a-4ea1-a794-08e0c0eafeb2\" (UID: \"6f061b05-d54a-4ea1-a794-08e0c0eafeb2\") "
Oct 30 19:41:10.141649 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:10.141534    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6f061b05-d54a-4ea1-a794-08e0c0eafeb2-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6f061b05-d54a-4ea1-a794-08e0c0eafeb2-maas-api-xks-ca\") pod \"6f061b05-d54a-4ea1-a794-08e0c0eafeb2\" (UID: \"6f061b05-d54a-4ea1-a794-08e0c0eafeb2\") "
Oct 30 19:41:10.153490 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:10.147835    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f061b05-d54a-4ea1-a794-08e0c0eafeb2-maas-api-openshift-ca" pod "6f061b05-d54a-4ea1-a794-08e0c0eafeb2" (UID: "6f061b05-d54a-4ea1-a794-08e0c0eafeb2"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:41:10.154078 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:10.153973    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f061b05-d54a-4ea1-a794-08e0c0eafeb2-maas-api-xks-ca" pod "6f061b05-d54a-4ea1-a794-08e0c0eafeb2" (UID: "6f061b05-d54a-4ea1-a794-08e0c0eafeb2"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:41:10.154366 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:10.154344    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f061b05-d54a-4ea1-a794-08e0c0eafeb2-kube-api-access-xd58t" pod "6f061b05-d54a-4ea1-a794-08e0c0eafeb2" (UID: "6f061b05-d54a-4ea1-a794-08e0c0eafeb2"). InnerVolumeSpecName "kube-api-access-xd58t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:41:10.245234 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:10.245199    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xd58t\" (UniqueName: \"kubernetes.io/projected/6f061b05-d54a-4ea1-a794-08e0c0eafeb2-kube-api-access-xd58t\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:10.245234 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:10.245235    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6f061b05-d54a-4ea1-a794-08e0c0eafeb2-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:10.245434 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:10.245250    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6f061b05-d54a-4ea1-a794-08e0c0eafeb2-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:10.294456 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:10.294418    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1e44b517-ce16-497d-8c07-bc9a56e00e41" path="/var/lib/kubelet/pods/1e44b517-ce16-497d-8c07-bc9a56e00e41/volumes"
Oct 30 19:41:10.321552 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:10.321456    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe4ec3-7944b894b6-6qx48" event={"ID":"e876e507-2b04-43fd-b788-b10d7d15748a","Type":"ContainerStarted","Data":"97e542a8cc024aa83523d5e6c8634887ffa5e67220bbd716fd4c10d77fd76ff0"}
Oct 30 19:41:10.321720 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:10.321565    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe4ec3-7944b894b6-6qx48"
Oct 30 19:41:10.324026 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:10.323995    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c6fd28cf-6rjhc" event={"ID":"6f061b05-d54a-4ea1-a794-08e0c0eafeb2","Type":"ContainerDied","Data":"38be2d75976f6e6d3c320359d340a4eb37edb233cda5716d5a593db0085fabc4"}
Oct 30 19:41:10.324026 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:10.324021    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c6fd28cf-6rjhc"
Oct 30 19:41:10.324216 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:10.324053    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="38be2d75976f6e6d3c320359d340a4eb37edb233cda5716d5a593db0085fabc4"
Oct 30 19:41:10.324539 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:10.324517    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:41:10.325693 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:10.325552    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:41:10.380892 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:10.380804    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe4ec3-7944b894b6-6qx48" podStartSLOduration=4.380785768 podStartE2EDuration="4.380785768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:41:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:41:10.372547783 +0000 UTC m=+1182.215479721" watchObservedRunningTime="2026-10-30 19:41:10.380785768 +0000 UTC m=+1182.223717708"
Oct 30 19:41:10.855290 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:10.855252    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c6fd28cf-6f49bf84bd-j4mh6"]
Oct 30 19:41:10.944490 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:10.944450    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c6fd28cf-667f98f88c-kstvs"]
Oct 30 19:41:12.588918 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:12.587846    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-sfh2c"
Oct 30 19:41:13.280929 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:13.233135    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-86a5394c-djb6k"]
Oct 30 19:41:13.299372 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:13.299320    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-86a5394c-djb6k"]
Oct 30 19:41:14.049200 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:14.049125    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f2c512a3-qh7tb"]
Oct 30 19:41:14.059184 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:14.059134    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f2c512a3-qh7tb"]
Oct 30 19:41:14.296950 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:14.296912    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4b948304-5c98-48e7-ad5a-345322b06f95" path="/var/lib/kubelet/pods/4b948304-5c98-48e7-ad5a-345322b06f95/volumes"
Oct 30 19:41:14.297746 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:14.297725    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e1c88445-d3b4-4fd8-9d68-98146be50d99" path="/var/lib/kubelet/pods/e1c88445-d3b4-4fd8-9d68-98146be50d99/volumes"
Oct 30 19:41:15.860993 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:15.860920    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-c6fd28cf-6f49bf84bd-j4mh6" podUID="5e15145e-9cae-4aba-9ba4-9fb31522485e" containerName="payload-processing" containerID="cri-o://d604bff502ec51978ce7d483266602b6605ba702ad06444eeb1ffb376365c0fc" gracePeriod=25
Oct 30 19:41:15.945583 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:15.945510    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6fd28cf-667f98f88c-kstvs" podUID="04b9bd1e-b30d-4d4f-b045-9441ce2415f7" containerName="payload-pre-processing" containerID="cri-o://3784b140c5972dc88af50e5259a98ca0b4920517bf9cd461e0082f96591adeda" gracePeriod=25
Oct 30 19:41:16.282614 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:16.282119    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c6fd28cf-6f49bf84bd-j4mh6"
Oct 30 19:41:16.827371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:16.375730    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6fd28cf-667f98f88c-kstvs"
Oct 30 19:41:16.827371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:16.424332    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5e15145e-9cae-4aba-9ba4-9fb31522485e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e15145e-9cae-4aba-9ba4-9fb31522485e-plugins-config-volume\") pod \"5e15145e-9cae-4aba-9ba4-9fb31522485e\" (UID: \"5e15145e-9cae-4aba-9ba4-9fb31522485e\") "
Oct 30 19:41:16.827371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:16.424728    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e15145e-9cae-4aba-9ba4-9fb31522485e-plugins-config-volume" pod "5e15145e-9cae-4aba-9ba4-9fb31522485e" (UID: "5e15145e-9cae-4aba-9ba4-9fb31522485e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:41:16.827371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:16.424989    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e15145e-9cae-4aba-9ba4-9fb31522485e-kube-api-access-m426j\" (UniqueName: \"kubernetes.io/projected/5e15145e-9cae-4aba-9ba4-9fb31522485e-kube-api-access-m426j\") pod \"5e15145e-9cae-4aba-9ba4-9fb31522485e\" (UID: \"5e15145e-9cae-4aba-9ba4-9fb31522485e\") "
Oct 30 19:41:16.827371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:16.426512    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e15145e-9cae-4aba-9ba4-9fb31522485e-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:16.827371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:16.429337    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e15145e-9cae-4aba-9ba4-9fb31522485e-kube-api-access-m426j" pod "5e15145e-9cae-4aba-9ba4-9fb31522485e" (UID: "5e15145e-9cae-4aba-9ba4-9fb31522485e"). InnerVolumeSpecName "kube-api-access-m426j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:41:16.827371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:16.504033    5907 generic.go:356] "Generic (PLEG): container finished" podID="04b9bd1e-b30d-4d4f-b045-9441ce2415f7" containerID="3784b140c5972dc88af50e5259a98ca0b4920517bf9cd461e0082f96591adeda" exitCode=0
Oct 30 19:41:16.827371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:16.504241    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6fd28cf-667f98f88c-kstvs"
Oct 30 19:41:16.827371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:16.504658    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6fd28cf-667f98f88c-kstvs" event={"ID":"04b9bd1e-b30d-4d4f-b045-9441ce2415f7","Type":"ContainerDied","Data":"3784b140c5972dc88af50e5259a98ca0b4920517bf9cd461e0082f96591adeda"}
Oct 30 19:41:16.827371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:16.504715    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6fd28cf-667f98f88c-kstvs" event={"ID":"04b9bd1e-b30d-4d4f-b045-9441ce2415f7","Type":"ContainerDied","Data":"614ab8de7d825ae7486d3b81b9eaec1606577e6aeecfc0de1f8893db792e7af1"}
Oct 30 19:41:16.827371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:16.504745    5907 scope.go:122] "RemoveContainer" containerID="3784b140c5972dc88af50e5259a98ca0b4920517bf9cd461e0082f96591adeda"
Oct 30 19:41:16.827371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:16.507947    5907 generic.go:356] "Generic (PLEG): container finished" podID="5e15145e-9cae-4aba-9ba4-9fb31522485e" containerID="d604bff502ec51978ce7d483266602b6605ba702ad06444eeb1ffb376365c0fc" exitCode=0
Oct 30 19:41:16.827371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:16.507997    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c6fd28cf-6f49bf84bd-j4mh6" event={"ID":"5e15145e-9cae-4aba-9ba4-9fb31522485e","Type":"ContainerDied","Data":"d604bff502ec51978ce7d483266602b6605ba702ad06444eeb1ffb376365c0fc"}
Oct 30 19:41:16.827371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:16.508044    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c6fd28cf-6f49bf84bd-j4mh6" event={"ID":"5e15145e-9cae-4aba-9ba4-9fb31522485e","Type":"ContainerDied","Data":"b6fa54f0111789560040089ee84ea81fbe1382c0e67cf92e3bb01da8880cf6ec"}
Oct 30 19:41:16.827371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:16.508051    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c6fd28cf-6f49bf84bd-j4mh6"
Oct 30 19:41:16.827371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:16.527466    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/04b9bd1e-b30d-4d4f-b045-9441ce2415f7-kube-api-access-nx4tb\" (UniqueName: \"kubernetes.io/projected/04b9bd1e-b30d-4d4f-b045-9441ce2415f7-kube-api-access-nx4tb\") pod \"04b9bd1e-b30d-4d4f-b045-9441ce2415f7\" (UID: \"04b9bd1e-b30d-4d4f-b045-9441ce2415f7\") "
Oct 30 19:41:16.827371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:16.527722    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/04b9bd1e-b30d-4d4f-b045-9441ce2415f7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04b9bd1e-b30d-4d4f-b045-9441ce2415f7-plugins-config-volume\") pod \"04b9bd1e-b30d-4d4f-b045-9441ce2415f7\" (UID: \"04b9bd1e-b30d-4d4f-b045-9441ce2415f7\") "
Oct 30 19:41:16.827371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:16.528629    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04b9bd1e-b30d-4d4f-b045-9441ce2415f7-plugins-config-volume" pod "04b9bd1e-b30d-4d4f-b045-9441ce2415f7" (UID: "04b9bd1e-b30d-4d4f-b045-9441ce2415f7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:41:16.827371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:16.529018    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04b9bd1e-b30d-4d4f-b045-9441ce2415f7-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:16.827371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:16.529041    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m426j\" (UniqueName: \"kubernetes.io/projected/5e15145e-9cae-4aba-9ba4-9fb31522485e-kube-api-access-m426j\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:16.827371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:16.530668    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04b9bd1e-b30d-4d4f-b045-9441ce2415f7-kube-api-access-nx4tb" pod "04b9bd1e-b30d-4d4f-b045-9441ce2415f7" (UID: "04b9bd1e-b30d-4d4f-b045-9441ce2415f7"). InnerVolumeSpecName "kube-api-access-nx4tb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:41:16.827371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:16.533402    5907 scope.go:122] "RemoveContainer" containerID="3784b140c5972dc88af50e5259a98ca0b4920517bf9cd461e0082f96591adeda"
Oct 30 19:41:16.827371 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:41:16.533761    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3784b140c5972dc88af50e5259a98ca0b4920517bf9cd461e0082f96591adeda\": container with ID starting with 3784b140c5972dc88af50e5259a98ca0b4920517bf9cd461e0082f96591adeda not found: ID does not exist" containerID="3784b140c5972dc88af50e5259a98ca0b4920517bf9cd461e0082f96591adeda"
Oct 30 19:41:16.827371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:16.533796    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3784b140c5972dc88af50e5259a98ca0b4920517bf9cd461e0082f96591adeda"} err="failed to get container status \"3784b140c5972dc88af50e5259a98ca0b4920517bf9cd461e0082f96591adeda\": rpc error: code = NotFound desc = could not find container \"3784b140c5972dc88af50e5259a98ca0b4920517bf9cd461e0082f96591adeda\": container with ID starting with 3784b140c5972dc88af50e5259a98ca0b4920517bf9cd461e0082f96591adeda not found: ID does not exist"
Oct 30 19:41:16.827371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:16.533821    5907 scope.go:122] "RemoveContainer" containerID="d604bff502ec51978ce7d483266602b6605ba702ad06444eeb1ffb376365c0fc"
Oct 30 19:41:16.827371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:16.545708    5907 scope.go:122] "RemoveContainer" containerID="d604bff502ec51978ce7d483266602b6605ba702ad06444eeb1ffb376365c0fc"
Oct 30 19:41:16.827371 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:41:16.546108    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d604bff502ec51978ce7d483266602b6605ba702ad06444eeb1ffb376365c0fc\": container with ID starting with d604bff502ec51978ce7d483266602b6605ba702ad06444eeb1ffb376365c0fc not found: ID does not exist" containerID="d604bff502ec51978ce7d483266602b6605ba702ad06444eeb1ffb376365c0fc"
Oct 30 19:41:16.827371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:16.546135    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d604bff502ec51978ce7d483266602b6605ba702ad06444eeb1ffb376365c0fc"} err="failed to get container status \"d604bff502ec51978ce7d483266602b6605ba702ad06444eeb1ffb376365c0fc\": rpc error: code = NotFound desc = could not find container \"d604bff502ec51978ce7d483266602b6605ba702ad06444eeb1ffb376365c0fc\": container with ID starting with d604bff502ec51978ce7d483266602b6605ba702ad06444eeb1ffb376365c0fc not found: ID does not exist"
Oct 30 19:41:16.827371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:16.593774    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c6fd28cf-6f49bf84bd-j4mh6"]
Oct 30 19:41:16.827371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:16.601603    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c6fd28cf-6f49bf84bd-j4mh6"]
Oct 30 19:41:16.827371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:16.631850    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nx4tb\" (UniqueName: \"kubernetes.io/projected/04b9bd1e-b30d-4d4f-b045-9441ce2415f7-kube-api-access-nx4tb\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:16.856619 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:16.856595    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c6fd28cf-667f98f88c-kstvs"]
Oct 30 19:41:16.863336 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:16.863307    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c6fd28cf-667f98f88c-kstvs"]
Oct 30 19:41:17.504094 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:41:17.504058    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:41:17.523998 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:17.523963    5907 generic.go:356] "Generic (PLEG): container finished" podID="ec8c99fb-671c-4aea-b30e-543bc0e3d6c1" containerID="b37856698cdd9aa3a02cf54ddf8eaacf36ce6bd93693522ce0733e0b689dda9e" exitCode=0
Oct 30 19:41:17.524313 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:17.524008    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9" event={"ID":"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1","Type":"ContainerDied","Data":"b37856698cdd9aa3a02cf54ddf8eaacf36ce6bd93693522ce0733e0b689dda9e"}
Oct 30 19:41:17.524530 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:17.524503    5907 scope.go:122] "RemoveContainer" containerID="b37856698cdd9aa3a02cf54ddf8eaacf36ce6bd93693522ce0733e0b689dda9e"
Oct 30 19:41:17.526780 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:17.526747    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:41:17.526955 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:17.526875    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:41:17.526955 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:17.526926    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:41:17.530419 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:17.530373    5907 generic.go:356] "Generic (PLEG): container finished" podID="76a6fce4-3ba8-434a-b126-8a1a8465b1bf" containerID="00f49ab6f04e5d256b9f1fd0b6d2d75b2448c8aab2a7c1828d2e8c119788b7fb" exitCode=0
Oct 30 19:41:17.530544 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:17.530432    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6" event={"ID":"76a6fce4-3ba8-434a-b126-8a1a8465b1bf","Type":"ContainerDied","Data":"00f49ab6f04e5d256b9f1fd0b6d2d75b2448c8aab2a7c1828d2e8c119788b7fb"}
Oct 30 19:41:17.531056 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:17.531028    5907 scope.go:122] "RemoveContainer" containerID="00f49ab6f04e5d256b9f1fd0b6d2d75b2448c8aab2a7c1828d2e8c119788b7fb"
Oct 30 19:41:17.532613 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:17.532587    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:41:17.532696 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:17.532649    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:41:17.532747 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:17.532694    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:41:18.296667 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:18.296632    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="04b9bd1e-b30d-4d4f-b045-9441ce2415f7" path="/var/lib/kubelet/pods/04b9bd1e-b30d-4d4f-b045-9441ce2415f7/volumes"
Oct 30 19:41:18.297410 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:18.297388    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5e15145e-9cae-4aba-9ba4-9fb31522485e" path="/var/lib/kubelet/pods/5e15145e-9cae-4aba-9ba4-9fb31522485e/volumes"
Oct 30 19:41:19.028892 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:19.009271    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6" event={"ID":"76a6fce4-3ba8-434a-b126-8a1a8465b1bf","Type":"ContainerStarted","Data":"c298eb1f658ad6239263f90a8ee68905b2504b8c78bfb5a34b3fe987eeb766ea"}
Oct 30 19:41:19.315988 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:19.314447    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9" event={"ID":"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1","Type":"ContainerStarted","Data":"5587a38ceec70e314bc2ec1f0bcf07aadb8e043b79045e68681402543bd319ff"}
Oct 30 19:41:19.412886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:19.405221    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:41:19.445898 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:19.436705    5907 patch_prober.go:29] interesting pod/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused" start-of-body=
Oct 30 19:41:19.445898 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:19.436781    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6" podUID="76a6fce4-3ba8-434a-b126-8a1a8465b1bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused"
Oct 30 19:41:19.830886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:19.814930    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:41:19.830886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:19.815054    5907 patch_prober.go:29] interesting pod/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused" start-of-body=
Oct 30 19:41:19.830886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:19.815098    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9" podUID="ec8c99fb-671c-4aea-b30e-543bc0e3d6c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused"
Oct 30 19:41:19.866835 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:19.866802    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:41:20.009002 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:20.008965    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c6fd28cf-6rjhc"]
Oct 30 19:41:20.037976 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:20.037934    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c6fd28cf-6rjhc"]
Oct 30 19:41:20.066801 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:20.066765    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4202bb04-nn2r6"]
Oct 30 19:41:20.092025 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:20.079730    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4202bb04-nn2r6"]
Oct 30 19:41:20.415887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:20.410320    5907 patch_prober.go:29] interesting pod/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused" start-of-body=
Oct 30 19:41:20.415887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:20.410387    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6" podUID="76a6fce4-3ba8-434a-b126-8a1a8465b1bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused"
Oct 30 19:41:20.496899 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:20.496736    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="27c3c568-8107-46bf-8c4a-c35d7b549e4b" path="/var/lib/kubelet/pods/27c3c568-8107-46bf-8c4a-c35d7b549e4b/volumes"
Oct 30 19:41:20.497883 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:20.497537    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6f061b05-d54a-4ea1-a794-08e0c0eafeb2" path="/var/lib/kubelet/pods/6f061b05-d54a-4ea1-a794-08e0c0eafeb2/volumes"
Oct 30 19:41:20.834706 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:20.834614    5907 patch_prober.go:29] interesting pod/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused" start-of-body=
Oct 30 19:41:20.835016 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:20.834985    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9" podUID="ec8c99fb-671c-4aea-b30e-543bc0e3d6c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused"
Oct 30 19:41:21.396313 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:21.396280    5907 patch_prober.go:29] interesting pod/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused" start-of-body=
Oct 30 19:41:21.396600 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:21.396572    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6" podUID="76a6fce4-3ba8-434a-b126-8a1a8465b1bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused"
Oct 30 19:41:21.822053 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:21.821960    5907 patch_prober.go:29] interesting pod/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused" start-of-body=
Oct 30 19:41:21.822650 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:21.822615    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9" podUID="ec8c99fb-671c-4aea-b30e-543bc0e3d6c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused"
Oct 30 19:41:22.408892 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:22.408361    5907 patch_prober.go:29] interesting pod/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused" start-of-body=
Oct 30 19:41:22.408892 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:22.408447    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6" podUID="76a6fce4-3ba8-434a-b126-8a1a8465b1bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused"
Oct 30 19:41:22.524905 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:22.523056    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe4ec3-7944b894b6-6qx48"
Oct 30 19:41:22.605834 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:22.605786    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:41:22.876746 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:22.876710    5907 patch_prober.go:29] interesting pod/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused" start-of-body=
Oct 30 19:41:22.877372 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:22.877336    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9" podUID="ec8c99fb-671c-4aea-b30e-543bc0e3d6c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused"
Oct 30 19:41:23.396140 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:23.396111    5907 patch_prober.go:29] interesting pod/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused" start-of-body=
Oct 30 19:41:23.396366 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:23.396345    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6" podUID="76a6fce4-3ba8-434a-b126-8a1a8465b1bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused"
Oct 30 19:41:23.815767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:23.815256    5907 patch_prober.go:29] interesting pod/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused" start-of-body=
Oct 30 19:41:23.815767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:23.815332    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9" podUID="ec8c99fb-671c-4aea-b30e-543bc0e3d6c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused"
Oct 30 19:41:24.397067 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:24.397030    5907 patch_prober.go:29] interesting pod/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused" start-of-body=
Oct 30 19:41:24.397505 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:24.397087    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6" podUID="76a6fce4-3ba8-434a-b126-8a1a8465b1bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused"
Oct 30 19:41:24.815937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:24.815235    5907 patch_prober.go:29] interesting pod/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused" start-of-body=
Oct 30 19:41:24.815937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:24.815297    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9" podUID="ec8c99fb-671c-4aea-b30e-543bc0e3d6c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused"
Oct 30 19:41:25.402071 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:25.402044    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:41:25.403228 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:25.403204    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:41:25.834539 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:25.834453    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:41:25.835726 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:25.835703    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:41:26.251794 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:26.251719    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2495c2e8-xt2t2"]
Oct 30 19:41:26.335205 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:26.335167    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e308a20f-vskcl"]
Oct 30 19:41:26.340974 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:26.340944    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2495c2e8-xt2t2"]
Oct 30 19:41:26.345810 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:26.345780    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e308a20f-vskcl"]
Oct 30 19:41:28.292804 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:28.292774    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5fee739b-3a5d-4517-8b8a-ea26a4de9006" path="/var/lib/kubelet/pods/5fee739b-3a5d-4517-8b8a-ea26a4de9006/volumes"
Oct 30 19:41:28.293481 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:28.293457    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="987a4548-687f-46ee-bf92-0d9657ed2151" path="/var/lib/kubelet/pods/987a4548-687f-46ee-bf92-0d9657ed2151/volumes"
Oct 30 19:41:29.727543 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:29.727507    5907 scope.go:122] "RemoveContainer" containerID="cc0d24da48a6975020a162fb2520b9c890b27e8eb96b9547e1c6704cfe030587"
Oct 30 19:41:29.743809 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:29.743783    5907 scope.go:122] "RemoveContainer" containerID="75da0da1e21c466295bfa91185043b5bc76a29fb0ab995c9c9ce4940b4406cd0"
Oct 30 19:41:29.754972 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:29.754950    5907 scope.go:122] "RemoveContainer" containerID="1e8aef43e3c5d0eef98b32608a2f285a79c96935b99f73b6f9f90dcef681d2e0"
Oct 30 19:41:30.305927 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:30.305891    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"]
Oct 30 19:41:30.306209 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:30.306162    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw" podUID="61a95e57-7f30-4985-8efc-9cbbc0843993" containerName="istio-proxy" containerID="cri-o://28b18ab7383989113aadfd005d80ea891749b0449b6592e79c7e4efaf1ccac6f" gracePeriod=30
Oct 30 19:41:31.326540 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:31.326507    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"]
Oct 30 19:41:31.327002 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:31.326727    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn" podUID="43db2eba-9244-4db6-ae67-8a20b01ea53e" containerName="istio-proxy" containerID="cri-o://161dbeed1b1a6cfe0da1d965482bb0878c9ed0c873dfdf879651affd2560f856" gracePeriod=30
Oct 30 19:41:31.339559 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:31.339527    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"]
Oct 30 19:41:31.339828 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:31.339802    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c" podUID="452bcb07-8aa6-4e70-8f3a-af8bfaf756a4" containerName="istio-proxy" containerID="cri-o://3808fa08aab4443d03b99268d763f7d40e24c80c9eb5d899e96a30e6870e932c" gracePeriod=30
Oct 30 19:41:31.344962 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:31.344940    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"]
Oct 30 19:41:31.345172 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:31.345131    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8" podUID="3504b2cf-8f1c-453e-9372-3d425261b29c" containerName="istio-proxy" containerID="cri-o://8794bae25ae02f8e1d05acd7c30647daed122dbc6bb4cbfe0543a5724d29aa38" gracePeriod=30
Oct 30 19:41:31.654624 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:31.654590    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:41:31.654780 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:31.654647    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn" podUID="43db2eba-9244-4db6-ae67-8a20b01ea53e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:41:31.808144 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:31.808082    5907 patch_prober.go:29] interesting pod/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:41:31.808344 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:31.808153    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8" podUID="3504b2cf-8f1c-453e-9372-3d425261b29c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:41:35.630685 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.630660    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:41:35.780580 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.780436    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61a95e57-7f30-4985-8efc-9cbbc0843993-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61a95e57-7f30-4985-8efc-9cbbc0843993-workload-socket\") pod \"61a95e57-7f30-4985-8efc-9cbbc0843993\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") "
Oct 30 19:41:35.780942 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.780915    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/61a95e57-7f30-4985-8efc-9cbbc0843993-kube-api-access-sjlxn\" (UniqueName: \"kubernetes.io/projected/61a95e57-7f30-4985-8efc-9cbbc0843993-kube-api-access-sjlxn\") pod \"61a95e57-7f30-4985-8efc-9cbbc0843993\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") "
Oct 30 19:41:35.781222 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.781185    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61a95e57-7f30-4985-8efc-9cbbc0843993-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61a95e57-7f30-4985-8efc-9cbbc0843993-workload-certs\") pod \"61a95e57-7f30-4985-8efc-9cbbc0843993\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") "
Oct 30 19:41:35.781537 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.781516    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/61a95e57-7f30-4985-8efc-9cbbc0843993-istio-token\" (UniqueName: \"kubernetes.io/projected/61a95e57-7f30-4985-8efc-9cbbc0843993-istio-token\") pod \"61a95e57-7f30-4985-8efc-9cbbc0843993\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") "
Oct 30 19:41:35.781718 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.781699    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/61a95e57-7f30-4985-8efc-9cbbc0843993-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61a95e57-7f30-4985-8efc-9cbbc0843993-istiod-ca-cert\") pod \"61a95e57-7f30-4985-8efc-9cbbc0843993\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") "
Oct 30 19:41:35.781928 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.781910    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61a95e57-7f30-4985-8efc-9cbbc0843993-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61a95e57-7f30-4985-8efc-9cbbc0843993-credential-socket\") pod \"61a95e57-7f30-4985-8efc-9cbbc0843993\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") "
Oct 30 19:41:35.782137 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.782068    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61a95e57-7f30-4985-8efc-9cbbc0843993-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61a95e57-7f30-4985-8efc-9cbbc0843993-istio-data\") pod \"61a95e57-7f30-4985-8efc-9cbbc0843993\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") "
Oct 30 19:41:35.782319 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.782294    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61a95e57-7f30-4985-8efc-9cbbc0843993-workload-socket" pod "61a95e57-7f30-4985-8efc-9cbbc0843993" (UID: "61a95e57-7f30-4985-8efc-9cbbc0843993"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:41:35.782319 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.782299    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/61a95e57-7f30-4985-8efc-9cbbc0843993-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61a95e57-7f30-4985-8efc-9cbbc0843993-istio-podinfo\") pod \"61a95e57-7f30-4985-8efc-9cbbc0843993\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") "
Oct 30 19:41:35.782511 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.782492    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61a95e57-7f30-4985-8efc-9cbbc0843993-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61a95e57-7f30-4985-8efc-9cbbc0843993-istio-envoy\") pod \"61a95e57-7f30-4985-8efc-9cbbc0843993\" (UID: \"61a95e57-7f30-4985-8efc-9cbbc0843993\") "
Oct 30 19:41:35.784524 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.784220    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61a95e57-7f30-4985-8efc-9cbbc0843993-workload-certs" pod "61a95e57-7f30-4985-8efc-9cbbc0843993" (UID: "61a95e57-7f30-4985-8efc-9cbbc0843993"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:41:35.784524 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.784324    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61a95e57-7f30-4985-8efc-9cbbc0843993-credential-socket" pod "61a95e57-7f30-4985-8efc-9cbbc0843993" (UID: "61a95e57-7f30-4985-8efc-9cbbc0843993"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:41:35.784524 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.784481    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61a95e57-7f30-4985-8efc-9cbbc0843993-istio-data" pod "61a95e57-7f30-4985-8efc-9cbbc0843993" (UID: "61a95e57-7f30-4985-8efc-9cbbc0843993"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:41:35.784737 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.784532    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61a95e57-7f30-4985-8efc-9cbbc0843993-istiod-ca-cert" pod "61a95e57-7f30-4985-8efc-9cbbc0843993" (UID: "61a95e57-7f30-4985-8efc-9cbbc0843993"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:41:35.786785 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.785215    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61a95e57-7f30-4985-8efc-9cbbc0843993-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:35.786785 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.785238    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61a95e57-7f30-4985-8efc-9cbbc0843993-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:35.786785 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.785251    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61a95e57-7f30-4985-8efc-9cbbc0843993-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:35.786785 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.785263    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61a95e57-7f30-4985-8efc-9cbbc0843993-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:35.786785 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.785278    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61a95e57-7f30-4985-8efc-9cbbc0843993-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:35.791213 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.788777    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61a95e57-7f30-4985-8efc-9cbbc0843993-kube-api-access-sjlxn" pod "61a95e57-7f30-4985-8efc-9cbbc0843993" (UID: "61a95e57-7f30-4985-8efc-9cbbc0843993"). InnerVolumeSpecName "kube-api-access-sjlxn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:41:35.791213 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.788907    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/61a95e57-7f30-4985-8efc-9cbbc0843993-istio-podinfo" pod "61a95e57-7f30-4985-8efc-9cbbc0843993" (UID: "61a95e57-7f30-4985-8efc-9cbbc0843993"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:41:35.791213 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.790229    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61a95e57-7f30-4985-8efc-9cbbc0843993-istio-token" pod "61a95e57-7f30-4985-8efc-9cbbc0843993" (UID: "61a95e57-7f30-4985-8efc-9cbbc0843993"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:41:35.793049 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.793027    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61a95e57-7f30-4985-8efc-9cbbc0843993-istio-envoy" pod "61a95e57-7f30-4985-8efc-9cbbc0843993" (UID: "61a95e57-7f30-4985-8efc-9cbbc0843993"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:41:35.886768 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.886736    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61a95e57-7f30-4985-8efc-9cbbc0843993-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:35.886768 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.886767    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61a95e57-7f30-4985-8efc-9cbbc0843993-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:35.886981 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.886781    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61a95e57-7f30-4985-8efc-9cbbc0843993-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:35.886981 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.886795    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjlxn\" (UniqueName: \"kubernetes.io/projected/61a95e57-7f30-4985-8efc-9cbbc0843993-kube-api-access-sjlxn\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:35.946617 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.946580    5907 generic.go:356] "Generic (PLEG): container finished" podID="61a95e57-7f30-4985-8efc-9cbbc0843993" containerID="28b18ab7383989113aadfd005d80ea891749b0449b6592e79c7e4efaf1ccac6f" exitCode=0
Oct 30 19:41:35.946816 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.946634    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"
Oct 30 19:41:35.946816 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.946662    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw" event={"ID":"61a95e57-7f30-4985-8efc-9cbbc0843993","Type":"ContainerDied","Data":"28b18ab7383989113aadfd005d80ea891749b0449b6592e79c7e4efaf1ccac6f"}
Oct 30 19:41:35.946816 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.946721    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw" event={"ID":"61a95e57-7f30-4985-8efc-9cbbc0843993","Type":"ContainerDied","Data":"9a05aaaee8c0885cfe6b0d00e8308a9994a70c2869f98139f6fb26a8ca8bd24b"}
Oct 30 19:41:35.946816 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.946744    5907 scope.go:122] "RemoveContainer" containerID="28b18ab7383989113aadfd005d80ea891749b0449b6592e79c7e4efaf1ccac6f"
Oct 30 19:41:35.969758 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.969729    5907 scope.go:122] "RemoveContainer" containerID="f856dc1ad44a1d324020a8bc3c92782d6505bcffef001671948f0d800580d12a"
Oct 30 19:41:35.981801 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.981779    5907 scope.go:122] "RemoveContainer" containerID="28b18ab7383989113aadfd005d80ea891749b0449b6592e79c7e4efaf1ccac6f"
Oct 30 19:41:35.982100 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:41:35.982078    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28b18ab7383989113aadfd005d80ea891749b0449b6592e79c7e4efaf1ccac6f\": container with ID starting with 28b18ab7383989113aadfd005d80ea891749b0449b6592e79c7e4efaf1ccac6f not found: ID does not exist" containerID="28b18ab7383989113aadfd005d80ea891749b0449b6592e79c7e4efaf1ccac6f"
Oct 30 19:41:35.982191 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.982112    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28b18ab7383989113aadfd005d80ea891749b0449b6592e79c7e4efaf1ccac6f"} err="failed to get container status \"28b18ab7383989113aadfd005d80ea891749b0449b6592e79c7e4efaf1ccac6f\": rpc error: code = NotFound desc = could not find container \"28b18ab7383989113aadfd005d80ea891749b0449b6592e79c7e4efaf1ccac6f\": container with ID starting with 28b18ab7383989113aadfd005d80ea891749b0449b6592e79c7e4efaf1ccac6f not found: ID does not exist"
Oct 30 19:41:35.982191 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.982136    5907 scope.go:122] "RemoveContainer" containerID="f856dc1ad44a1d324020a8bc3c92782d6505bcffef001671948f0d800580d12a"
Oct 30 19:41:35.982416 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:41:35.982395    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f856dc1ad44a1d324020a8bc3c92782d6505bcffef001671948f0d800580d12a\": container with ID starting with f856dc1ad44a1d324020a8bc3c92782d6505bcffef001671948f0d800580d12a not found: ID does not exist" containerID="f856dc1ad44a1d324020a8bc3c92782d6505bcffef001671948f0d800580d12a"
Oct 30 19:41:35.982499 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:35.982421    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f856dc1ad44a1d324020a8bc3c92782d6505bcffef001671948f0d800580d12a"} err="failed to get container status \"f856dc1ad44a1d324020a8bc3c92782d6505bcffef001671948f0d800580d12a\": rpc error: code = NotFound desc = could not find container \"f856dc1ad44a1d324020a8bc3c92782d6505bcffef001671948f0d800580d12a\": container with ID starting with f856dc1ad44a1d324020a8bc3c92782d6505bcffef001671948f0d800580d12a not found: ID does not exist"
Oct 30 19:41:36.006528 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.006501    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"]
Oct 30 19:41:36.019123 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.019099    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-c6fd28cf-openshift-default-7bbb559c5c-d5kkw"]
Oct 30 19:41:36.049313 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.049279    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1cf2851b-ss6cg"]
Oct 30 19:41:36.054495 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.054470    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1cf2851b-ss6cg"]
Oct 30 19:41:36.291388 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.291312    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="61a95e57-7f30-4985-8efc-9cbbc0843993" path="/var/lib/kubelet/pods/61a95e57-7f30-4985-8efc-9cbbc0843993/volumes"
Oct 30 19:41:36.291914 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.291899    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a1e438ce-ee16-4984-8960-4a998d9515dc" path="/var/lib/kubelet/pods/a1e438ce-ee16-4984-8960-4a998d9515dc/volumes"
Oct 30 19:41:36.696116 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.696084    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"
Oct 30 19:41:36.700550 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.700526    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"
Oct 30 19:41:36.701111 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.701087    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/43db2eba-9244-4db6-ae67-8a20b01ea53e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43db2eba-9244-4db6-ae67-8a20b01ea53e-istio-data\") pod \"43db2eba-9244-4db6-ae67-8a20b01ea53e\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") "
Oct 30 19:41:36.701254 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.701158    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/43db2eba-9244-4db6-ae67-8a20b01ea53e-istio-token\" (UniqueName: \"kubernetes.io/projected/43db2eba-9244-4db6-ae67-8a20b01ea53e-istio-token\") pod \"43db2eba-9244-4db6-ae67-8a20b01ea53e\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") "
Oct 30 19:41:36.701412 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.701274    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/43db2eba-9244-4db6-ae67-8a20b01ea53e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43db2eba-9244-4db6-ae67-8a20b01ea53e-istio-podinfo\") pod \"43db2eba-9244-4db6-ae67-8a20b01ea53e\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") "
Oct 30 19:41:36.701412 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.701314    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/43db2eba-9244-4db6-ae67-8a20b01ea53e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43db2eba-9244-4db6-ae67-8a20b01ea53e-istiod-ca-cert\") pod \"43db2eba-9244-4db6-ae67-8a20b01ea53e\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") "
Oct 30 19:41:36.701412 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.701398    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/43db2eba-9244-4db6-ae67-8a20b01ea53e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43db2eba-9244-4db6-ae67-8a20b01ea53e-credential-socket\") pod \"43db2eba-9244-4db6-ae67-8a20b01ea53e\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") "
Oct 30 19:41:36.701598 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.701484    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/43db2eba-9244-4db6-ae67-8a20b01ea53e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43db2eba-9244-4db6-ae67-8a20b01ea53e-workload-socket\") pod \"43db2eba-9244-4db6-ae67-8a20b01ea53e\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") "
Oct 30 19:41:36.701598 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.701562    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/43db2eba-9244-4db6-ae67-8a20b01ea53e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43db2eba-9244-4db6-ae67-8a20b01ea53e-workload-certs\") pod \"43db2eba-9244-4db6-ae67-8a20b01ea53e\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") "
Oct 30 19:41:36.701598 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.701332    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43db2eba-9244-4db6-ae67-8a20b01ea53e-istio-data" pod "43db2eba-9244-4db6-ae67-8a20b01ea53e" (UID: "43db2eba-9244-4db6-ae67-8a20b01ea53e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:41:36.701759 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.701675    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/43db2eba-9244-4db6-ae67-8a20b01ea53e-kube-api-access-6vn6n\" (UniqueName: \"kubernetes.io/projected/43db2eba-9244-4db6-ae67-8a20b01ea53e-kube-api-access-6vn6n\") pod \"43db2eba-9244-4db6-ae67-8a20b01ea53e\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") "
Oct 30 19:41:36.701759 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.701738    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/43db2eba-9244-4db6-ae67-8a20b01ea53e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43db2eba-9244-4db6-ae67-8a20b01ea53e-istio-envoy\") pod \"43db2eba-9244-4db6-ae67-8a20b01ea53e\" (UID: \"43db2eba-9244-4db6-ae67-8a20b01ea53e\") "
Oct 30 19:41:36.701884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.701814    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43db2eba-9244-4db6-ae67-8a20b01ea53e-credential-socket" pod "43db2eba-9244-4db6-ae67-8a20b01ea53e" (UID: "43db2eba-9244-4db6-ae67-8a20b01ea53e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:41:36.702054 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.701936    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43db2eba-9244-4db6-ae67-8a20b01ea53e-istiod-ca-cert" pod "43db2eba-9244-4db6-ae67-8a20b01ea53e" (UID: "43db2eba-9244-4db6-ae67-8a20b01ea53e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:41:36.702123 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.702020    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43db2eba-9244-4db6-ae67-8a20b01ea53e-workload-certs" pod "43db2eba-9244-4db6-ae67-8a20b01ea53e" (UID: "43db2eba-9244-4db6-ae67-8a20b01ea53e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:41:36.702181 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.702163    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43db2eba-9244-4db6-ae67-8a20b01ea53e-workload-socket" pod "43db2eba-9244-4db6-ae67-8a20b01ea53e" (UID: "43db2eba-9244-4db6-ae67-8a20b01ea53e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:41:36.702733 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.702712    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43db2eba-9244-4db6-ae67-8a20b01ea53e-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:36.702902 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.702736    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43db2eba-9244-4db6-ae67-8a20b01ea53e-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:36.702902 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.702752    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43db2eba-9244-4db6-ae67-8a20b01ea53e-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:36.702902 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.702766    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43db2eba-9244-4db6-ae67-8a20b01ea53e-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:36.702902 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.702778    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43db2eba-9244-4db6-ae67-8a20b01ea53e-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:36.706278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.706254    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43db2eba-9244-4db6-ae67-8a20b01ea53e-kube-api-access-6vn6n" pod "43db2eba-9244-4db6-ae67-8a20b01ea53e" (UID: "43db2eba-9244-4db6-ae67-8a20b01ea53e"). InnerVolumeSpecName "kube-api-access-6vn6n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:41:36.707431 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.707407    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43db2eba-9244-4db6-ae67-8a20b01ea53e-istio-token" pod "43db2eba-9244-4db6-ae67-8a20b01ea53e" (UID: "43db2eba-9244-4db6-ae67-8a20b01ea53e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:41:36.707990 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.707964    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43db2eba-9244-4db6-ae67-8a20b01ea53e-istio-envoy" pod "43db2eba-9244-4db6-ae67-8a20b01ea53e" (UID: "43db2eba-9244-4db6-ae67-8a20b01ea53e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:41:36.708665 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.708641    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/43db2eba-9244-4db6-ae67-8a20b01ea53e-istio-podinfo" pod "43db2eba-9244-4db6-ae67-8a20b01ea53e" (UID: "43db2eba-9244-4db6-ae67-8a20b01ea53e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:41:36.742429 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.742407    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"
Oct 30 19:41:36.803480 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.803445    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3504b2cf-8f1c-453e-9372-3d425261b29c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3504b2cf-8f1c-453e-9372-3d425261b29c-istio-podinfo\") pod \"3504b2cf-8f1c-453e-9372-3d425261b29c\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") "
Oct 30 19:41:36.803642 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.803502    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3504b2cf-8f1c-453e-9372-3d425261b29c-kube-api-access-pc4dj\" (UniqueName: \"kubernetes.io/projected/3504b2cf-8f1c-453e-9372-3d425261b29c-kube-api-access-pc4dj\") pod \"3504b2cf-8f1c-453e-9372-3d425261b29c\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") "
Oct 30 19:41:36.803642 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.803531    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3504b2cf-8f1c-453e-9372-3d425261b29c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3504b2cf-8f1c-453e-9372-3d425261b29c-istiod-ca-cert\") pod \"3504b2cf-8f1c-453e-9372-3d425261b29c\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") "
Oct 30 19:41:36.803642 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.803563    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-istio-envoy\") pod \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") "
Oct 30 19:41:36.803642 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.803582    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-workload-certs\") pod \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") "
Oct 30 19:41:36.803912 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.803704    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-kube-api-access-ncwc7\" (UniqueName: \"kubernetes.io/projected/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-kube-api-access-ncwc7\") pod \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") "
Oct 30 19:41:36.803912 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.803801    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3504b2cf-8f1c-453e-9372-3d425261b29c-istio-token\" (UniqueName: \"kubernetes.io/projected/3504b2cf-8f1c-453e-9372-3d425261b29c-istio-token\") pod \"3504b2cf-8f1c-453e-9372-3d425261b29c\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") "
Oct 30 19:41:36.803912 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.803839    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-istio-token\" (UniqueName: \"kubernetes.io/projected/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-istio-token\") pod \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") "
Oct 30 19:41:36.804102 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.803969    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-workload-socket\") pod \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") "
Oct 30 19:41:36.804102 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.804066    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3504b2cf-8f1c-453e-9372-3d425261b29c-istiod-ca-cert" pod "3504b2cf-8f1c-453e-9372-3d425261b29c" (UID: "3504b2cf-8f1c-453e-9372-3d425261b29c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:41:36.804218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.804155    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3504b2cf-8f1c-453e-9372-3d425261b29c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3504b2cf-8f1c-453e-9372-3d425261b29c-workload-socket\") pod \"3504b2cf-8f1c-453e-9372-3d425261b29c\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") "
Oct 30 19:41:36.804218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.804188    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3504b2cf-8f1c-453e-9372-3d425261b29c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3504b2cf-8f1c-453e-9372-3d425261b29c-workload-certs\") pod \"3504b2cf-8f1c-453e-9372-3d425261b29c\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") "
Oct 30 19:41:36.804326 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.804282    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-workload-certs" pod "452bcb07-8aa6-4e70-8f3a-af8bfaf756a4" (UID: "452bcb07-8aa6-4e70-8f3a-af8bfaf756a4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:41:36.804466 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.804447    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-credential-socket\") pod \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") "
Oct 30 19:41:36.804536 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.804507    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3504b2cf-8f1c-453e-9372-3d425261b29c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3504b2cf-8f1c-453e-9372-3d425261b29c-credential-socket\") pod \"3504b2cf-8f1c-453e-9372-3d425261b29c\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") "
Oct 30 19:41:36.804590 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.804539    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-istio-podinfo\") pod \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") "
Oct 30 19:41:36.804590 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.804553    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-workload-socket" pod "452bcb07-8aa6-4e70-8f3a-af8bfaf756a4" (UID: "452bcb07-8aa6-4e70-8f3a-af8bfaf756a4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:41:36.804590 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.804568    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-istiod-ca-cert\") pod \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") "
Oct 30 19:41:36.804755 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.804601    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3504b2cf-8f1c-453e-9372-3d425261b29c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3504b2cf-8f1c-453e-9372-3d425261b29c-istio-envoy\") pod \"3504b2cf-8f1c-453e-9372-3d425261b29c\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") "
Oct 30 19:41:36.804755 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.804708    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3504b2cf-8f1c-453e-9372-3d425261b29c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3504b2cf-8f1c-453e-9372-3d425261b29c-istio-data\") pod \"3504b2cf-8f1c-453e-9372-3d425261b29c\" (UID: \"3504b2cf-8f1c-453e-9372-3d425261b29c\") "
Oct 30 19:41:36.804755 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.804742    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-credential-socket" pod "452bcb07-8aa6-4e70-8f3a-af8bfaf756a4" (UID: "452bcb07-8aa6-4e70-8f3a-af8bfaf756a4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:41:36.804978 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.804800    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-istio-data\") pod \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\" (UID: \"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4\") "
Oct 30 19:41:36.804978 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.804954    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3504b2cf-8f1c-453e-9372-3d425261b29c-workload-socket" pod "3504b2cf-8f1c-453e-9372-3d425261b29c" (UID: "3504b2cf-8f1c-453e-9372-3d425261b29c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:41:36.805513 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.805138    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3504b2cf-8f1c-453e-9372-3d425261b29c-workload-certs" pod "3504b2cf-8f1c-453e-9372-3d425261b29c" (UID: "3504b2cf-8f1c-453e-9372-3d425261b29c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:41:36.805513 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.805340    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-istio-data" pod "452bcb07-8aa6-4e70-8f3a-af8bfaf756a4" (UID: "452bcb07-8aa6-4e70-8f3a-af8bfaf756a4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:41:36.805700 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.805520    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3504b2cf-8f1c-453e-9372-3d425261b29c-credential-socket" pod "3504b2cf-8f1c-453e-9372-3d425261b29c" (UID: "3504b2cf-8f1c-453e-9372-3d425261b29c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:41:36.806462 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.806362    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:36.806462 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.806398    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vn6n\" (UniqueName: \"kubernetes.io/projected/43db2eba-9244-4db6-ae67-8a20b01ea53e-kube-api-access-6vn6n\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:36.806462 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.806418    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3504b2cf-8f1c-453e-9372-3d425261b29c-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:36.806462 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.806434    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43db2eba-9244-4db6-ae67-8a20b01ea53e-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:36.806462 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.806449    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:36.806462 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.806463    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:36.806462 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.806478    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43db2eba-9244-4db6-ae67-8a20b01ea53e-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:36.806462 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.806493    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3504b2cf-8f1c-453e-9372-3d425261b29c-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:36.807064 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.806508    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3504b2cf-8f1c-453e-9372-3d425261b29c-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:36.807064 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.806505    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3504b2cf-8f1c-453e-9372-3d425261b29c-istio-podinfo" pod "3504b2cf-8f1c-453e-9372-3d425261b29c" (UID: "3504b2cf-8f1c-453e-9372-3d425261b29c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:41:36.807064 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.806523    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43db2eba-9244-4db6-ae67-8a20b01ea53e-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:36.807064 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.806539    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:36.807064 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.806554    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3504b2cf-8f1c-453e-9372-3d425261b29c-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:36.807064 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.806608    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-istio-envoy" pod "452bcb07-8aa6-4e70-8f3a-af8bfaf756a4" (UID: "452bcb07-8aa6-4e70-8f3a-af8bfaf756a4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:41:36.807064 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.806851    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3504b2cf-8f1c-453e-9372-3d425261b29c-istio-data" pod "3504b2cf-8f1c-453e-9372-3d425261b29c" (UID: "3504b2cf-8f1c-453e-9372-3d425261b29c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:41:36.807432 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.807343    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-istiod-ca-cert" pod "452bcb07-8aa6-4e70-8f3a-af8bfaf756a4" (UID: "452bcb07-8aa6-4e70-8f3a-af8bfaf756a4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:41:36.807617 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.807593    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-kube-api-access-ncwc7" pod "452bcb07-8aa6-4e70-8f3a-af8bfaf756a4" (UID: "452bcb07-8aa6-4e70-8f3a-af8bfaf756a4"). InnerVolumeSpecName "kube-api-access-ncwc7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:41:36.807712 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.807641    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3504b2cf-8f1c-453e-9372-3d425261b29c-kube-api-access-pc4dj" pod "3504b2cf-8f1c-453e-9372-3d425261b29c" (UID: "3504b2cf-8f1c-453e-9372-3d425261b29c"). InnerVolumeSpecName "kube-api-access-pc4dj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:41:36.807955 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.807933    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-istio-podinfo" pod "452bcb07-8aa6-4e70-8f3a-af8bfaf756a4" (UID: "452bcb07-8aa6-4e70-8f3a-af8bfaf756a4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:41:36.808120 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.808092    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-istio-token" pod "452bcb07-8aa6-4e70-8f3a-af8bfaf756a4" (UID: "452bcb07-8aa6-4e70-8f3a-af8bfaf756a4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:41:36.808255 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.808154    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3504b2cf-8f1c-453e-9372-3d425261b29c-istio-token" pod "3504b2cf-8f1c-453e-9372-3d425261b29c" (UID: "3504b2cf-8f1c-453e-9372-3d425261b29c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:41:36.808840 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.808815    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3504b2cf-8f1c-453e-9372-3d425261b29c-istio-envoy" pod "3504b2cf-8f1c-453e-9372-3d425261b29c" (UID: "3504b2cf-8f1c-453e-9372-3d425261b29c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:41:36.908781 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.908701    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:36.908781 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.908729    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:36.908781 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.908738    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3504b2cf-8f1c-453e-9372-3d425261b29c-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:36.908781 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.908745    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3504b2cf-8f1c-453e-9372-3d425261b29c-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:36.908781 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.908753    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3504b2cf-8f1c-453e-9372-3d425261b29c-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:36.908781 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.908762    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pc4dj\" (UniqueName: \"kubernetes.io/projected/3504b2cf-8f1c-453e-9372-3d425261b29c-kube-api-access-pc4dj\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:36.908781 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.908770    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:36.908781 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.908779    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ncwc7\" (UniqueName: \"kubernetes.io/projected/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-kube-api-access-ncwc7\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:36.908781 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.908787    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3504b2cf-8f1c-453e-9372-3d425261b29c-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:36.909220 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.908796    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:41:36.959006 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.958969    5907 generic.go:356] "Generic (PLEG): container finished" podID="452bcb07-8aa6-4e70-8f3a-af8bfaf756a4" containerID="3808fa08aab4443d03b99268d763f7d40e24c80c9eb5d899e96a30e6870e932c" exitCode=0
Oct 30 19:41:36.959192 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.959032    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"
Oct 30 19:41:36.959192 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.959051    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c" event={"ID":"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4","Type":"ContainerDied","Data":"3808fa08aab4443d03b99268d763f7d40e24c80c9eb5d899e96a30e6870e932c"}
Oct 30 19:41:36.959192 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.959104    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c" event={"ID":"452bcb07-8aa6-4e70-8f3a-af8bfaf756a4","Type":"ContainerDied","Data":"49d2b426df4d957089e4502d27e2064ecfb1422ef9e00b746ec43facee77db06"}
Oct 30 19:41:36.959192 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.959135    5907 scope.go:122] "RemoveContainer" containerID="3808fa08aab4443d03b99268d763f7d40e24c80c9eb5d899e96a30e6870e932c"
Oct 30 19:41:36.961540 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.961487    5907 generic.go:356] "Generic (PLEG): container finished" podID="43db2eba-9244-4db6-ae67-8a20b01ea53e" containerID="161dbeed1b1a6cfe0da1d965482bb0878c9ed0c873dfdf879651affd2560f856" exitCode=0
Oct 30 19:41:36.961700 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.961596    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn" event={"ID":"43db2eba-9244-4db6-ae67-8a20b01ea53e","Type":"ContainerDied","Data":"161dbeed1b1a6cfe0da1d965482bb0878c9ed0c873dfdf879651affd2560f856"}
Oct 30 19:41:36.961700 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.961637    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn" event={"ID":"43db2eba-9244-4db6-ae67-8a20b01ea53e","Type":"ContainerDied","Data":"866028d1062316c1d10d3396e0c0c8c7cc877c3775baae13fe8f5ac1517f5c16"}
Oct 30 19:41:36.961839 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.961704    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"
Oct 30 19:41:36.963819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.963765    5907 generic.go:356] "Generic (PLEG): container finished" podID="3504b2cf-8f1c-453e-9372-3d425261b29c" containerID="8794bae25ae02f8e1d05acd7c30647daed122dbc6bb4cbfe0543a5724d29aa38" exitCode=0
Oct 30 19:41:36.963819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.963785    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"
Oct 30 19:41:36.964017 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.963812    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8" event={"ID":"3504b2cf-8f1c-453e-9372-3d425261b29c","Type":"ContainerDied","Data":"8794bae25ae02f8e1d05acd7c30647daed122dbc6bb4cbfe0543a5724d29aa38"}
Oct 30 19:41:36.964017 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.963883    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8" event={"ID":"3504b2cf-8f1c-453e-9372-3d425261b29c","Type":"ContainerDied","Data":"dfa5e0ee1d238417fbc064aeabc0bd472aac2eb554a712e508adb9a752d1c1af"}
Oct 30 19:41:36.973586 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.973569    5907 scope.go:122] "RemoveContainer" containerID="3808fa08aab4443d03b99268d763f7d40e24c80c9eb5d899e96a30e6870e932c"
Oct 30 19:41:36.973949 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:41:36.973920    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3808fa08aab4443d03b99268d763f7d40e24c80c9eb5d899e96a30e6870e932c\": container with ID starting with 3808fa08aab4443d03b99268d763f7d40e24c80c9eb5d899e96a30e6870e932c not found: ID does not exist" containerID="3808fa08aab4443d03b99268d763f7d40e24c80c9eb5d899e96a30e6870e932c"
Oct 30 19:41:36.974130 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.973960    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3808fa08aab4443d03b99268d763f7d40e24c80c9eb5d899e96a30e6870e932c"} err="failed to get container status \"3808fa08aab4443d03b99268d763f7d40e24c80c9eb5d899e96a30e6870e932c\": rpc error: code = NotFound desc = could not find container \"3808fa08aab4443d03b99268d763f7d40e24c80c9eb5d899e96a30e6870e932c\": container with ID starting with 3808fa08aab4443d03b99268d763f7d40e24c80c9eb5d899e96a30e6870e932c not found: ID does not exist"
Oct 30 19:41:36.974130 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.974043    5907 scope.go:122] "RemoveContainer" containerID="161dbeed1b1a6cfe0da1d965482bb0878c9ed0c873dfdf879651affd2560f856"
Oct 30 19:41:36.989568 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.989520    5907 scope.go:122] "RemoveContainer" containerID="161dbeed1b1a6cfe0da1d965482bb0878c9ed0c873dfdf879651affd2560f856"
Oct 30 19:41:36.990034 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:41:36.990007    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"161dbeed1b1a6cfe0da1d965482bb0878c9ed0c873dfdf879651affd2560f856\": container with ID starting with 161dbeed1b1a6cfe0da1d965482bb0878c9ed0c873dfdf879651affd2560f856 not found: ID does not exist" containerID="161dbeed1b1a6cfe0da1d965482bb0878c9ed0c873dfdf879651affd2560f856"
Oct 30 19:41:36.990146 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.990043    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"161dbeed1b1a6cfe0da1d965482bb0878c9ed0c873dfdf879651affd2560f856"} err="failed to get container status \"161dbeed1b1a6cfe0da1d965482bb0878c9ed0c873dfdf879651affd2560f856\": rpc error: code = NotFound desc = could not find container \"161dbeed1b1a6cfe0da1d965482bb0878c9ed0c873dfdf879651affd2560f856\": container with ID starting with 161dbeed1b1a6cfe0da1d965482bb0878c9ed0c873dfdf879651affd2560f856 not found: ID does not exist"
Oct 30 19:41:36.990146 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:36.990067    5907 scope.go:122] "RemoveContainer" containerID="8794bae25ae02f8e1d05acd7c30647daed122dbc6bb4cbfe0543a5724d29aa38"
Oct 30 19:41:37.000764 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:37.000734    5907 scope.go:122] "RemoveContainer" containerID="8794bae25ae02f8e1d05acd7c30647daed122dbc6bb4cbfe0543a5724d29aa38"
Oct 30 19:41:37.001041 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:41:37.001014    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8794bae25ae02f8e1d05acd7c30647daed122dbc6bb4cbfe0543a5724d29aa38\": container with ID starting with 8794bae25ae02f8e1d05acd7c30647daed122dbc6bb4cbfe0543a5724d29aa38 not found: ID does not exist" containerID="8794bae25ae02f8e1d05acd7c30647daed122dbc6bb4cbfe0543a5724d29aa38"
Oct 30 19:41:37.001102 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:37.001047    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8794bae25ae02f8e1d05acd7c30647daed122dbc6bb4cbfe0543a5724d29aa38"} err="failed to get container status \"8794bae25ae02f8e1d05acd7c30647daed122dbc6bb4cbfe0543a5724d29aa38\": rpc error: code = NotFound desc = could not find container \"8794bae25ae02f8e1d05acd7c30647daed122dbc6bb4cbfe0543a5724d29aa38\": container with ID starting with 8794bae25ae02f8e1d05acd7c30647daed122dbc6bb4cbfe0543a5724d29aa38 not found: ID does not exist"
Oct 30 19:41:37.041735 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:37.041709    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"]
Oct 30 19:41:37.046361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:37.046334    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j42z8"]
Oct 30 19:41:37.089226 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:37.089198    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"]
Oct 30 19:41:37.104303 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:37.104277    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-j6f9c"]
Oct 30 19:41:37.203603 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:37.203512    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"]
Oct 30 19:41:37.215266 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:37.215225    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-gcsfn"]
Oct 30 19:41:38.290385 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.290357    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3504b2cf-8f1c-453e-9372-3d425261b29c" path="/var/lib/kubelet/pods/3504b2cf-8f1c-453e-9372-3d425261b29c/volumes"
Oct 30 19:41:38.291001 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.290985    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="43db2eba-9244-4db6-ae67-8a20b01ea53e" path="/var/lib/kubelet/pods/43db2eba-9244-4db6-ae67-8a20b01ea53e/volumes"
Oct 30 19:41:38.291465 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.291452    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="452bcb07-8aa6-4e70-8f3a-af8bfaf756a4" path="/var/lib/kubelet/pods/452bcb07-8aa6-4e70-8f3a-af8bfaf756a4/volumes"
Oct 30 19:41:38.510048 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.510015    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4a4e2f-5f844996c5-8zh8m"]
Oct 30 19:41:38.510805 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.510791    5907 cpu_manager.go:414] "containerMap: removing container" podUID="3504b2cf-8f1c-453e-9372-3d425261b29c" containerName="istio-proxy"
Oct 30 19:41:38.510854 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.510807    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3504b2cf-8f1c-453e-9372-3d425261b29c" containerName="istio-proxy"
Oct 30 19:41:38.510854 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.510843    5907 cpu_manager.go:414] "containerMap: removing container" podUID="61a95e57-7f30-4985-8efc-9cbbc0843993" containerName="istio-proxy"
Oct 30 19:41:38.510854 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.510851    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61a95e57-7f30-4985-8efc-9cbbc0843993" containerName="istio-proxy"
Oct 30 19:41:38.510962 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.510894    5907 cpu_manager.go:414] "containerMap: removing container" podUID="6f061b05-d54a-4ea1-a794-08e0c0eafeb2" containerName="revoke-keys"
Oct 30 19:41:38.510962 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.510902    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6f061b05-d54a-4ea1-a794-08e0c0eafeb2" containerName="revoke-keys"
Oct 30 19:41:38.510962 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.510915    5907 cpu_manager.go:414] "containerMap: removing container" podUID="5e15145e-9cae-4aba-9ba4-9fb31522485e" containerName="payload-processing"
Oct 30 19:41:38.510962 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.510922    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5e15145e-9cae-4aba-9ba4-9fb31522485e" containerName="payload-processing"
Oct 30 19:41:38.510962 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.510936    5907 cpu_manager.go:414] "containerMap: removing container" podUID="04b9bd1e-b30d-4d4f-b045-9441ce2415f7" containerName="payload-pre-processing"
Oct 30 19:41:38.510962 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.510940    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="04b9bd1e-b30d-4d4f-b045-9441ce2415f7" containerName="payload-pre-processing"
Oct 30 19:41:38.510962 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.510955    5907 cpu_manager.go:414] "containerMap: removing container" podUID="452bcb07-8aa6-4e70-8f3a-af8bfaf756a4" containerName="istio-proxy"
Oct 30 19:41:38.510962 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.510960    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="452bcb07-8aa6-4e70-8f3a-af8bfaf756a4" containerName="istio-proxy"
Oct 30 19:41:38.511200 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.510974    5907 cpu_manager.go:414] "containerMap: removing container" podUID="43db2eba-9244-4db6-ae67-8a20b01ea53e" containerName="istio-proxy"
Oct 30 19:41:38.511200 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.510979    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="43db2eba-9244-4db6-ae67-8a20b01ea53e" containerName="istio-proxy"
Oct 30 19:41:38.511200 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.510985    5907 cpu_manager.go:414] "containerMap: removing container" podUID="61a95e57-7f30-4985-8efc-9cbbc0843993" containerName="istio-proxy"
Oct 30 19:41:38.511200 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.510989    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61a95e57-7f30-4985-8efc-9cbbc0843993" containerName="istio-proxy"
Oct 30 19:41:38.511200 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.511131    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="3504b2cf-8f1c-453e-9372-3d425261b29c" containerName="istio-proxy"
Oct 30 19:41:38.511200 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.511143    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="61a95e57-7f30-4985-8efc-9cbbc0843993" containerName="istio-proxy"
Oct 30 19:41:38.511200 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.511151    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="6f061b05-d54a-4ea1-a794-08e0c0eafeb2" containerName="revoke-keys"
Oct 30 19:41:38.511200 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.511156    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="5e15145e-9cae-4aba-9ba4-9fb31522485e" containerName="payload-processing"
Oct 30 19:41:38.511200 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.511163    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="04b9bd1e-b30d-4d4f-b045-9441ce2415f7" containerName="payload-pre-processing"
Oct 30 19:41:38.511200 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.511171    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="452bcb07-8aa6-4e70-8f3a-af8bfaf756a4" containerName="istio-proxy"
Oct 30 19:41:38.511200 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.511177    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="43db2eba-9244-4db6-ae67-8a20b01ea53e" containerName="istio-proxy"
Oct 30 19:41:38.511200 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.511182    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="61a95e57-7f30-4985-8efc-9cbbc0843993" containerName="istio-proxy"
Oct 30 19:41:38.516717 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.516696    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a4e2f-5f844996c5-8zh8m"
Oct 30 19:41:38.551022 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.550954    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-4a4e2f-dockercfg-xsqhl\""
Oct 30 19:41:38.551263 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.551065    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-4a4e2f\""
Oct 30 19:41:38.553221 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.553193    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rh2x\" (UniqueName: \"kubernetes.io/projected/347a693d-f331-4300-9bec-d9ee3fafa5a8-kube-api-access-9rh2x\") pod \"payload-pre-processing-e2e-api-a-4a4e2f-5f844996c5-8zh8m\" (UID: \"347a693d-f331-4300-9bec-d9ee3fafa5a8\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a4e2f-5f844996c5-8zh8m"
Oct 30 19:41:38.553470 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.553450    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/347a693d-f331-4300-9bec-d9ee3fafa5a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-4a4e2f-5f844996c5-8zh8m\" (UID: \"347a693d-f331-4300-9bec-d9ee3fafa5a8\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a4e2f-5f844996c5-8zh8m"
Oct 30 19:41:38.558119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.558094    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4a4e2f-5f844996c5-8zh8m"]
Oct 30 19:41:38.669888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.664458    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rh2x\" (UniqueName: \"kubernetes.io/projected/347a693d-f331-4300-9bec-d9ee3fafa5a8-kube-api-access-9rh2x\") pod \"payload-pre-processing-e2e-api-a-4a4e2f-5f844996c5-8zh8m\" (UID: \"347a693d-f331-4300-9bec-d9ee3fafa5a8\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a4e2f-5f844996c5-8zh8m"
Oct 30 19:41:38.669888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.664758    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/347a693d-f331-4300-9bec-d9ee3fafa5a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-4a4e2f-5f844996c5-8zh8m\" (UID: \"347a693d-f331-4300-9bec-d9ee3fafa5a8\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a4e2f-5f844996c5-8zh8m"
Oct 30 19:41:38.669888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:38.665777    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/347a693d-f331-4300-9bec-d9ee3fafa5a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-4a4e2f-5f844996c5-8zh8m\" (UID: \"347a693d-f331-4300-9bec-d9ee3fafa5a8\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a4e2f-5f844996c5-8zh8m"
Oct 30 19:41:39.119116 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:39.117789    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rh2x\" (UniqueName: \"kubernetes.io/projected/347a693d-f331-4300-9bec-d9ee3fafa5a8-kube-api-access-9rh2x\") pod \"payload-pre-processing-e2e-api-a-4a4e2f-5f844996c5-8zh8m\" (UID: \"347a693d-f331-4300-9bec-d9ee3fafa5a8\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a4e2f-5f844996c5-8zh8m"
Oct 30 19:41:39.136748 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:39.136709    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4a4e2f-7dc687499d-xwj5f"]
Oct 30 19:41:39.137235 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:39.137205    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a4e2f-5f844996c5-8zh8m"
Oct 30 19:41:39.145667 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:39.145038    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4a4e2f-7dc687499d-xwj5f"
Oct 30 19:41:39.148601 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:39.148577    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4a4e2f-7dc687499d-xwj5f"]
Oct 30 19:41:39.219221 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:39.217674    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab-plugins-config-volume\") pod \"payload-processing-e2e-api-a-4a4e2f-7dc687499d-xwj5f\" (UID: \"5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab\") " pod="openshift-ingress/payload-processing-e2e-api-a-4a4e2f-7dc687499d-xwj5f"
Oct 30 19:41:39.219221 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:39.217761    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xngv\" (UniqueName: \"kubernetes.io/projected/5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab-kube-api-access-9xngv\") pod \"payload-processing-e2e-api-a-4a4e2f-7dc687499d-xwj5f\" (UID: \"5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab\") " pod="openshift-ingress/payload-processing-e2e-api-a-4a4e2f-7dc687499d-xwj5f"
Oct 30 19:41:39.319694 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:39.319663    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab-plugins-config-volume\") pod \"payload-processing-e2e-api-a-4a4e2f-7dc687499d-xwj5f\" (UID: \"5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab\") " pod="openshift-ingress/payload-processing-e2e-api-a-4a4e2f-7dc687499d-xwj5f"
Oct 30 19:41:39.320119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:39.319923    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xngv\" (UniqueName: \"kubernetes.io/projected/5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab-kube-api-access-9xngv\") pod \"payload-processing-e2e-api-a-4a4e2f-7dc687499d-xwj5f\" (UID: \"5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab\") " pod="openshift-ingress/payload-processing-e2e-api-a-4a4e2f-7dc687499d-xwj5f"
Oct 30 19:41:39.320681 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:39.320651    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab-plugins-config-volume\") pod \"payload-processing-e2e-api-a-4a4e2f-7dc687499d-xwj5f\" (UID: \"5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab\") " pod="openshift-ingress/payload-processing-e2e-api-a-4a4e2f-7dc687499d-xwj5f"
Oct 30 19:41:39.349658 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:39.349626    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xngv\" (UniqueName: \"kubernetes.io/projected/5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab-kube-api-access-9xngv\") pod \"payload-processing-e2e-api-a-4a4e2f-7dc687499d-xwj5f\" (UID: \"5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab\") " pod="openshift-ingress/payload-processing-e2e-api-a-4a4e2f-7dc687499d-xwj5f"
Oct 30 19:41:39.488648 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:39.488617    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4a4e2f-5f844996c5-8zh8m"]
Oct 30 19:41:39.491410 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:41:39.491383    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod347a693d_f331_4300_9bec_d9ee3fafa5a8.slice/crio-19b3cb611991e16e4b7a7c4e4aa3a400b3d35b51454d34685a1b7c5534d6c148 WatchSource:0}: Error finding container 19b3cb611991e16e4b7a7c4e4aa3a400b3d35b51454d34685a1b7c5534d6c148: Status 404 returned error can't find the container with id 19b3cb611991e16e4b7a7c4e4aa3a400b3d35b51454d34685a1b7c5534d6c148
Oct 30 19:41:39.492648 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:39.492625    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4a4e2f-7dc687499d-xwj5f"
Oct 30 19:41:39.885987 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:41:39.881982    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c3f5f70_da3c_4bf0_82fc_1453a7ebd0ab.slice/crio-f4482b4042841d1d68d3dbf8440341af94dbe2d4261bbfc3e8cad0ef486177b1 WatchSource:0}: Error finding container f4482b4042841d1d68d3dbf8440341af94dbe2d4261bbfc3e8cad0ef486177b1: Status 404 returned error can't find the container with id f4482b4042841d1d68d3dbf8440341af94dbe2d4261bbfc3e8cad0ef486177b1
Oct 30 19:41:39.957772 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:39.957726    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4a4e2f-7dc687499d-xwj5f"]
Oct 30 19:41:40.141395 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:40.141300    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4a4e2f-7dc687499d-xwj5f" event={"ID":"5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab","Type":"ContainerStarted","Data":"cf13beb2e4b2914a72f2074e59e59f53afa92bd927bd07a55c852230e8b106d2"}
Oct 30 19:41:40.141395 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:40.141359    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4a4e2f-7dc687499d-xwj5f" event={"ID":"5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab","Type":"ContainerStarted","Data":"f4482b4042841d1d68d3dbf8440341af94dbe2d4261bbfc3e8cad0ef486177b1"}
Oct 30 19:41:40.141395 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:40.141392    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-4a4e2f-7dc687499d-xwj5f"
Oct 30 19:41:40.143118 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:40.143081    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a4e2f-5f844996c5-8zh8m" event={"ID":"347a693d-f331-4300-9bec-d9ee3fafa5a8","Type":"ContainerStarted","Data":"e09a75759f5a2cbfd24fcf135899111c74dab123fb6204094cbaabd4d3516eab"}
Oct 30 19:41:40.143274 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:40.143124    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a4e2f-5f844996c5-8zh8m" event={"ID":"347a693d-f331-4300-9bec-d9ee3fafa5a8","Type":"ContainerStarted","Data":"19b3cb611991e16e4b7a7c4e4aa3a400b3d35b51454d34685a1b7c5534d6c148"}
Oct 30 19:41:40.143274 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:40.143193    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a4e2f-5f844996c5-8zh8m"
Oct 30 19:41:40.194621 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:40.194568    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-4a4e2f-7dc687499d-xwj5f" podStartSLOduration=2.194553101 podStartE2EDuration="2.194553101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:41:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:41:40.188205441 +0000 UTC m=+1212.031137379" watchObservedRunningTime="2026-10-30 19:41:40.194553101 +0000 UTC m=+1212.037485055"
Oct 30 19:41:40.242310 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:40.242255    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a4e2f-5f844996c5-8zh8m" podStartSLOduration=2.24224553 podStartE2EDuration="2.24224553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:41:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:41:40.241197401 +0000 UTC m=+1212.084129342" watchObservedRunningTime="2026-10-30 19:41:40.24224553 +0000 UTC m=+1212.085177468"
Oct 30 19:41:42.050173 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:42.049424    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-65fd42a4-8bv28"]
Oct 30 19:41:42.066112 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:42.066082    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-65fd42a4-8bv28"]
Oct 30 19:41:42.070347 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:42.070324    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-44b833ff-9zk88"]
Oct 30 19:41:42.074856 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:42.074827    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-44b833ff-9zk88"]
Oct 30 19:41:42.290663 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:42.290627    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c0228ac8-d5d3-4f74-8505-71d58292814a" path="/var/lib/kubelet/pods/c0228ac8-d5d3-4f74-8505-71d58292814a/volumes"
Oct 30 19:41:42.291169 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:42.291153    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d99e5ef4-84b7-4dd7-9251-b3aac9842f97" path="/var/lib/kubelet/pods/d99e5ef4-84b7-4dd7-9251-b3aac9842f97/volumes"
Oct 30 19:41:43.057636 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:43.057603    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7fbf748e-xx9cn"]
Oct 30 19:41:43.070181 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:43.070154    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7fbf748e-xx9cn"]
Oct 30 19:41:44.290456 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:44.290424    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5493aa92-3882-424c-b4dd-ab462198db29" path="/var/lib/kubelet/pods/5493aa92-3882-424c-b4dd-ab462198db29/volumes"
Oct 30 19:41:47.505617 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:41:47.505575    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:41:49.196953 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:49.187664    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:41:50.066948 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:50.066915    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-73ea5927-jmbxl"]
Oct 30 19:41:50.086991 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:50.086947    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-eeeb8238-8kv2x"]
Oct 30 19:41:50.093309 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:50.093275    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-73ea5927-jmbxl"]
Oct 30 19:41:50.096715 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:50.096682    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-38aef1fb-zvkv2"]
Oct 30 19:41:50.112422 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:50.112388    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-eeeb8238-8kv2x"]
Oct 30 19:41:50.116788 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:50.116762    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-38aef1fb-zvkv2"]
Oct 30 19:41:50.409143 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:50.409103    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="28e35538-2c5d-4ec4-8099-a7e7e6f8e270" path="/var/lib/kubelet/pods/28e35538-2c5d-4ec4-8099-a7e7e6f8e270/volumes"
Oct 30 19:41:50.410833 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:50.410807    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b347a63a-1983-4a46-aead-87af5078d844" path="/var/lib/kubelet/pods/b347a63a-1983-4a46-aead-87af5078d844/volumes"
Oct 30 19:41:50.411788 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:50.411765    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ce43f919-2435-4850-9436-3f7cc4863e4a" path="/var/lib/kubelet/pods/ce43f919-2435-4850-9436-3f7cc4863e4a/volumes"
Oct 30 19:41:51.056998 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:51.056961    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e87b29b8-l6d7c"]
Oct 30 19:41:51.067758 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:51.067721    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e87b29b8-l6d7c"]
Oct 30 19:41:51.190158 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:51.190134    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a4e2f-5f844996c5-8zh8m"
Oct 30 19:41:51.190345 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:51.190199    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-4a4e2f-7dc687499d-xwj5f"
Oct 30 19:41:52.290853 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:52.290822    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="27f2b187-607b-45f9-9d72-cbb2bfe1c81e" path="/var/lib/kubelet/pods/27f2b187-607b-45f9-9d72-cbb2bfe1c81e/volumes"
Oct 30 19:41:54.085895 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:54.081372    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-95a4820f-ng97l"]
Oct 30 19:41:54.100126 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:54.099480    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-95a4820f-ng97l"]
Oct 30 19:41:54.291294 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:54.291263    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dc5fa222-f6d2-4a77-9cc9-8a70bd322c95" path="/var/lib/kubelet/pods/dc5fa222-f6d2-4a77-9cc9-8a70bd322c95/volumes"
Oct 30 19:41:57.039810 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:57.039768    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8e42d71a-q6p8w"]
Oct 30 19:41:57.048640 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:57.048611    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c8b8d0ba-n2hgt"]
Oct 30 19:41:57.052549 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:57.052527    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8e42d71a-q6p8w"]
Oct 30 19:41:57.058639 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:57.058608    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c8b8d0ba-n2hgt"]
Oct 30 19:41:58.291464 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:58.291429    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="04d55fb9-1145-4041-bb56-e7b398104a74" path="/var/lib/kubelet/pods/04d55fb9-1145-4041-bb56-e7b398104a74/volumes"
Oct 30 19:41:58.291992 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:58.291976    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5" path="/var/lib/kubelet/pods/97e12df4-b6c1-4df4-bbd9-cdebfc4c57e5/volumes"
Oct 30 19:41:59.040176 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:59.040134    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e78dc66-ssgxc"]
Oct 30 19:41:59.047396 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:41:59.047367    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e78dc66-ssgxc"]
Oct 30 19:42:00.294304 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:00.294266    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bf7311e4-19fe-4dff-ba2f-e5c310c89c71" path="/var/lib/kubelet/pods/bf7311e4-19fe-4dff-ba2f-e5c310c89c71/volumes"
Oct 30 19:42:00.389531 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:00.389496    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"]
Oct 30 19:42:00.409942 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:00.408648    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"
Oct 30 19:42:00.430908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:00.430764    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"]
Oct 30 19:42:00.434849 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:00.434817    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-67953efd\"/\"openshift-service-ca.crt\""
Oct 30 19:42:00.439749 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:00.439728    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-67953efd\"/\"kube-root-ca.crt\""
Oct 30 19:42:00.439912 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:00.439763    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-67953efd\"/\"default-dockercfg-j2p49\""
Oct 30 19:42:00.439912 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:00.439808    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-67953efd\"/\"e2e-lifecycle-model-67953efd-kserve-self-signed-certs\""
Oct 30 19:42:00.480098 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:00.480061    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-dshm\") pod \"e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6\" (UID: \"c3f73773-6c2f-47e4-a5a4-c57916f64ac9\") " pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"
Oct 30 19:42:00.480289 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:00.480274    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnjd6\" (UniqueName: \"kubernetes.io/projected/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-kube-api-access-qnjd6\") pod \"e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6\" (UID: \"c3f73773-6c2f-47e4-a5a4-c57916f64ac9\") " pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"
Oct 30 19:42:00.480386 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:00.480373    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-home\") pod \"e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6\" (UID: \"c3f73773-6c2f-47e4-a5a4-c57916f64ac9\") " pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"
Oct 30 19:42:00.480441 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:00.480398    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-model-cache\") pod \"e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6\" (UID: \"c3f73773-6c2f-47e4-a5a4-c57916f64ac9\") " pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"
Oct 30 19:42:00.480478 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:00.480460    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-tmp-dir\") pod \"e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6\" (UID: \"c3f73773-6c2f-47e4-a5a4-c57916f64ac9\") " pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"
Oct 30 19:42:00.480512 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:00.480477    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-tls-certs\") pod \"e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6\" (UID: \"c3f73773-6c2f-47e4-a5a4-c57916f64ac9\") " pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"
Oct 30 19:42:00.581633 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:00.581536    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnjd6\" (UniqueName: \"kubernetes.io/projected/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-kube-api-access-qnjd6\") pod \"e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6\" (UID: \"c3f73773-6c2f-47e4-a5a4-c57916f64ac9\") " pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"
Oct 30 19:42:00.581799 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:00.581743    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-home\") pod \"e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6\" (UID: \"c3f73773-6c2f-47e4-a5a4-c57916f64ac9\") " pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"
Oct 30 19:42:00.581799 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:00.581781    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-model-cache\") pod \"e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6\" (UID: \"c3f73773-6c2f-47e4-a5a4-c57916f64ac9\") " pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"
Oct 30 19:42:00.582026 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:00.581845    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-tmp-dir\") pod \"e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6\" (UID: \"c3f73773-6c2f-47e4-a5a4-c57916f64ac9\") " pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"
Oct 30 19:42:00.582026 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:00.581895    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-tls-certs\") pod \"e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6\" (UID: \"c3f73773-6c2f-47e4-a5a4-c57916f64ac9\") " pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"
Oct 30 19:42:00.582026 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:00.582004    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-dshm\") pod \"e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6\" (UID: \"c3f73773-6c2f-47e4-a5a4-c57916f64ac9\") " pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"
Oct 30 19:42:00.582229 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:00.582193    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-home\") pod \"e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6\" (UID: \"c3f73773-6c2f-47e4-a5a4-c57916f64ac9\") " pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"
Oct 30 19:42:00.582313 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:00.582238    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-tmp-dir\") pod \"e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6\" (UID: \"c3f73773-6c2f-47e4-a5a4-c57916f64ac9\") " pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"
Oct 30 19:42:00.582558 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:00.582531    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-model-cache\") pod \"e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6\" (UID: \"c3f73773-6c2f-47e4-a5a4-c57916f64ac9\") " pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"
Oct 30 19:42:00.584656 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:00.584629    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-dshm\") pod \"e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6\" (UID: \"c3f73773-6c2f-47e4-a5a4-c57916f64ac9\") " pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"
Oct 30 19:42:00.585135 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:00.585110    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-tls-certs\") pod \"e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6\" (UID: \"c3f73773-6c2f-47e4-a5a4-c57916f64ac9\") " pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"
Oct 30 19:42:00.603485 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:00.603462    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnjd6\" (UniqueName: \"kubernetes.io/projected/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-kube-api-access-qnjd6\") pod \"e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6\" (UID: \"c3f73773-6c2f-47e4-a5a4-c57916f64ac9\") " pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"
Oct 30 19:42:00.765179 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:00.740684    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"
Oct 30 19:42:01.098946 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:01.095209    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4f6a322d-cs2r8"]
Oct 30 19:42:01.226971 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:01.212983    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4f6a322d-cs2r8"]
Oct 30 19:42:01.816735 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:01.816658    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"]
Oct 30 19:42:01.825035 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:42:01.825004    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc3f73773_6c2f_47e4_a5a4_c57916f64ac9.slice/crio-04d95f1cf9c35a651b50e1052b3b5b43ee28c35868e65b5013ad53225dd94c92 WatchSource:0}: Error finding container 04d95f1cf9c35a651b50e1052b3b5b43ee28c35868e65b5013ad53225dd94c92: Status 404 returned error can't find the container with id 04d95f1cf9c35a651b50e1052b3b5b43ee28c35868e65b5013ad53225dd94c92
Oct 30 19:42:02.292918 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:02.292876    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="588e2d37-9738-4f5f-ac0e-f4992ae7eb53" path="/var/lib/kubelet/pods/588e2d37-9738-4f5f-ac0e-f4992ae7eb53/volumes"
Oct 30 19:42:02.797611 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:02.797574    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6" event={"ID":"c3f73773-6c2f-47e4-a5a4-c57916f64ac9","Type":"ContainerStarted","Data":"73be7a140f7c340a40c5680df706a9beff4c8be230f47f0ee20ffb1ec73d3071"}
Oct 30 19:42:02.797611 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:02.797626    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6" event={"ID":"c3f73773-6c2f-47e4-a5a4-c57916f64ac9","Type":"ContainerStarted","Data":"04d95f1cf9c35a651b50e1052b3b5b43ee28c35868e65b5013ad53225dd94c92"}
Oct 30 19:42:02.846909 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:02.846805    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6" podStartSLOduration=2.846786983 podStartE2EDuration="2.846786983s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:42:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:42:02.843485199 +0000 UTC m=+1234.686417137" watchObservedRunningTime="2026-10-30 19:42:02.846786983 +0000 UTC m=+1234.689718923"
Oct 30 19:42:03.779894 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:03.770991    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"
Oct 30 19:42:04.865761 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:04.865718    5907 generic.go:356] "Generic (PLEG): container finished" podID="0435effb-f3d6-450f-beed-937cec7badb0" containerID="e26d6b861043f3fc8b434d4774de412d9c28fc362a35d3c2135c4569742f2e00" exitCode=0
Oct 30 19:42:04.895361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:04.867853    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8" event={"ID":"0435effb-f3d6-450f-beed-937cec7badb0","Type":"ContainerDied","Data":"e26d6b861043f3fc8b434d4774de412d9c28fc362a35d3c2135c4569742f2e00"}
Oct 30 19:42:04.896222 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:04.896192    5907 scope.go:122] "RemoveContainer" containerID="e26d6b861043f3fc8b434d4774de412d9c28fc362a35d3c2135c4569742f2e00"
Oct 30 19:42:04.943806 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:04.933722    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:42:04.943806 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:04.933825    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:42:04.943806 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:04.933907    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:42:05.765895 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:05.765085    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"]
Oct 30 19:42:05.787168 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:05.787136    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:42:05.823889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:05.805462    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-workload-certs\") pod \"e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") " pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:42:05.824222 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:05.824190    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-istio-token\") pod \"e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") " pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:42:05.915887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:05.899348    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-istio-data\") pod \"e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") " pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:42:05.915887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:05.899403    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-istio-podinfo\") pod \"e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") " pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:42:05.915887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:05.899448    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dndc4\" (UniqueName: \"kubernetes.io/projected/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-kube-api-access-dndc4\") pod \"e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") " pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:42:05.915887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:05.899547    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-credential-socket\") pod \"e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") " pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:42:05.915887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:05.899588    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-istiod-ca-cert\") pod \"e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") " pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:42:05.915887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:05.899844    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-istio-envoy\") pod \"e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") " pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:42:05.915887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:05.900000    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-workload-socket\") pod \"e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") " pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:42:05.915887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:05.827298    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w1-291d35-openshift-default-dockercfg-hmwd5\""
Oct 30 19:42:05.994820 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:05.991588    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"]
Oct 30 19:42:06.040585 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.036628    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-credential-socket\") pod \"e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") " pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:42:06.040585 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.036697    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-istiod-ca-cert\") pod \"e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") " pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:42:06.040585 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.036952    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-istio-envoy\") pod \"e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") " pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:42:06.040585 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.037040    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-workload-socket\") pod \"e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") " pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:42:06.040585 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.037227    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-workload-certs\") pod \"e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") " pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:42:06.040585 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.037282    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-istio-token\") pod \"e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") " pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:42:06.040585 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.037319    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-istio-data\") pod \"e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") " pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:42:06.040585 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.037349    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-istio-podinfo\") pod \"e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") " pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:42:06.040585 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.037394    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dndc4\" (UniqueName: \"kubernetes.io/projected/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-kube-api-access-dndc4\") pod \"e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") " pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:42:06.040585 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.038656    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-credential-socket\") pod \"e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") " pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:42:06.040585 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.039461    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-istiod-ca-cert\") pod \"e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") " pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:42:06.220949 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.206203    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-workload-socket\") pod \"e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") " pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:42:06.223445 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.221777    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-istio-envoy\") pod \"e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") " pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:42:06.223445 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.221984    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-istio-data\") pod \"e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") " pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:42:06.223445 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.223046    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-workload-certs\") pod \"e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") " pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:42:06.234231 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.233619    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-43eebade-tlhqf"]
Oct 30 19:42:06.245196 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.245166    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-43eebade-tlhqf"]
Oct 30 19:42:06.245683 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.245663    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-43eebade-tlhqf"
Oct 30 19:42:06.313571 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.312783    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:42:06.313571 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.313483    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:42:06.380889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.374640    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dndc4\" (UniqueName: \"kubernetes.io/projected/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-kube-api-access-dndc4\") pod \"e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") " pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:42:06.380889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.379819    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2c4t8\" (UniqueName: \"kubernetes.io/projected/a37f0676-70db-40df-a028-8edb736a903c-kube-api-access-2c4t8\") pod \"maas-api-revoke-sub-43eebade-tlhqf\" (UID: \"a37f0676-70db-40df-a028-8edb736a903c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-43eebade-tlhqf"
Oct 30 19:42:06.380889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.380049    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a37f0676-70db-40df-a028-8edb736a903c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-43eebade-tlhqf\" (UID: \"a37f0676-70db-40df-a028-8edb736a903c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-43eebade-tlhqf"
Oct 30 19:42:06.380889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.380201    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a37f0676-70db-40df-a028-8edb736a903c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-43eebade-tlhqf\" (UID: \"a37f0676-70db-40df-a028-8edb736a903c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-43eebade-tlhqf"
Oct 30 19:42:06.397623 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.397506    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-istio-podinfo\") pod \"e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") " pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:42:06.399237 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.399203    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-istio-token\") pod \"e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") " pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:42:06.404691 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.404569    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8" event={"ID":"0435effb-f3d6-450f-beed-937cec7badb0","Type":"ContainerStarted","Data":"4be7617af316401736a1464fcac5393879ac9028143df75d2beae64a22f2660d"}
Oct 30 19:42:06.482337 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.482305    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c4t8\" (UniqueName: \"kubernetes.io/projected/a37f0676-70db-40df-a028-8edb736a903c-kube-api-access-2c4t8\") pod \"maas-api-revoke-sub-43eebade-tlhqf\" (UID: \"a37f0676-70db-40df-a028-8edb736a903c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-43eebade-tlhqf"
Oct 30 19:42:06.482508 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.482367    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a37f0676-70db-40df-a028-8edb736a903c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-43eebade-tlhqf\" (UID: \"a37f0676-70db-40df-a028-8edb736a903c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-43eebade-tlhqf"
Oct 30 19:42:06.482508 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.482467    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a37f0676-70db-40df-a028-8edb736a903c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-43eebade-tlhqf\" (UID: \"a37f0676-70db-40df-a028-8edb736a903c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-43eebade-tlhqf"
Oct 30 19:42:06.483095 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.483066    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a37f0676-70db-40df-a028-8edb736a903c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-43eebade-tlhqf\" (UID: \"a37f0676-70db-40df-a028-8edb736a903c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-43eebade-tlhqf"
Oct 30 19:42:06.485210 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.485181    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a37f0676-70db-40df-a028-8edb736a903c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-43eebade-tlhqf\" (UID: \"a37f0676-70db-40df-a028-8edb736a903c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-43eebade-tlhqf"
Oct 30 19:42:06.504332 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.504306    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c4t8\" (UniqueName: \"kubernetes.io/projected/a37f0676-70db-40df-a028-8edb736a903c-kube-api-access-2c4t8\") pod \"maas-api-revoke-sub-43eebade-tlhqf\" (UID: \"a37f0676-70db-40df-a028-8edb736a903c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-43eebade-tlhqf"
Oct 30 19:42:06.534229 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.534199    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:42:06.534427 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.534284    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:42:06.535077 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.535056    5907 patch_prober.go:29] interesting pod/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused" start-of-body=
Oct 30 19:42:06.535156 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.535096    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8" podUID="0435effb-f3d6-450f-beed-937cec7badb0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused"
Oct 30 19:42:06.688873 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.688831    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-43eebade-tlhqf"
Oct 30 19:42:06.871954 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.871923    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"]
Oct 30 19:42:06.876589 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:42:06.876561    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc09cc6c_b5b2_4108_be0a_35d3c3f03ffe.slice/crio-23d66735b804293fe6188b28273c173f3a92c1ead051754efc5da507bd5b1ce1 WatchSource:0}: Error finding container 23d66735b804293fe6188b28273c173f3a92c1ead051754efc5da507bd5b1ce1: Status 404 returned error can't find the container with id 23d66735b804293fe6188b28273c173f3a92c1ead051754efc5da507bd5b1ce1
Oct 30 19:42:06.880009 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.879967    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:42:06.880121 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.880063    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:42:06.880192 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:06.880115    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:42:07.037293 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:07.037258    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-43eebade-tlhqf"]
Oct 30 19:42:07.348819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:07.348775    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd" event={"ID":"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe","Type":"ContainerStarted","Data":"45500cbe0d670569b588d9d75bd90ea5d62448e8a0c448d49058575d078886f5"}
Oct 30 19:42:07.349045 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:07.348842    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd" event={"ID":"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe","Type":"ContainerStarted","Data":"23d66735b804293fe6188b28273c173f3a92c1ead051754efc5da507bd5b1ce1"}
Oct 30 19:42:07.350844 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:07.350812    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-43eebade-tlhqf" event={"ID":"a37f0676-70db-40df-a028-8edb736a903c","Type":"ContainerStarted","Data":"e06bb93467f917dd9497ce7ef392ae89a56c322774ee81a51f6572b9060f6d4a"}
Oct 30 19:42:07.395064 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:07.395014    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd" podStartSLOduration=2.394999199 podStartE2EDuration="2.394999199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:42:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:42:07.393343657 +0000 UTC m=+1239.236275596" watchObservedRunningTime="2026-10-30 19:42:07.394999199 +0000 UTC m=+1239.237931138"
Oct 30 19:42:07.547555 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:07.540975    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:42:07.560066 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:07.560024    5907 patch_prober.go:29] interesting pod/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused" start-of-body=
Oct 30 19:42:07.560440 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:07.560396    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8" podUID="0435effb-f3d6-450f-beed-937cec7badb0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused"
Oct 30 19:42:07.661897 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:07.637460    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:42:08.374391 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:08.374346    5907 generic.go:356] "Generic (PLEG): container finished" podID="a37f0676-70db-40df-a028-8edb736a903c" containerID="0ec5e3e105c41201d3d8979a08a3631d8fedf4b6e741d0df86b238c049f44a11" exitCode=0
Oct 30 19:42:08.375232 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:08.375202    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-43eebade-tlhqf" event={"ID":"a37f0676-70db-40df-a028-8edb736a903c","Type":"ContainerDied","Data":"0ec5e3e105c41201d3d8979a08a3631d8fedf4b6e741d0df86b238c049f44a11"}
Oct 30 19:42:08.375782 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:08.375759    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:42:08.377012 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:08.376986    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:42:08.534940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:08.533763    5907 patch_prober.go:29] interesting pod/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused" start-of-body=
Oct 30 19:42:08.534940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:08.533831    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8" podUID="0435effb-f3d6-450f-beed-937cec7badb0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused"
Oct 30 19:42:09.541886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:09.540027    5907 patch_prober.go:29] interesting pod/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused" start-of-body=
Oct 30 19:42:09.541886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:09.540129    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8" podUID="0435effb-f3d6-450f-beed-937cec7badb0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused"
Oct 30 19:42:09.636890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:09.635957    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:42:09.751506 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:09.751390    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-43eebade-tlhqf"
Oct 30 19:42:09.999674 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:09.999577    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a37f0676-70db-40df-a028-8edb736a903c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a37f0676-70db-40df-a028-8edb736a903c-maas-api-openshift-ca\") pod \"a37f0676-70db-40df-a028-8edb736a903c\" (UID: \"a37f0676-70db-40df-a028-8edb736a903c\") "
Oct 30 19:42:09.999979 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:09.999935    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a37f0676-70db-40df-a028-8edb736a903c-kube-api-access-2c4t8\" (UniqueName: \"kubernetes.io/projected/a37f0676-70db-40df-a028-8edb736a903c-kube-api-access-2c4t8\") pod \"a37f0676-70db-40df-a028-8edb736a903c\" (UID: \"a37f0676-70db-40df-a028-8edb736a903c\") "
Oct 30 19:42:10.000205 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:10.000186    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a37f0676-70db-40df-a028-8edb736a903c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a37f0676-70db-40df-a028-8edb736a903c-maas-api-xks-ca\") pod \"a37f0676-70db-40df-a028-8edb736a903c\" (UID: \"a37f0676-70db-40df-a028-8edb736a903c\") "
Oct 30 19:42:10.001615 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:10.001581    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a37f0676-70db-40df-a028-8edb736a903c-maas-api-openshift-ca" pod "a37f0676-70db-40df-a028-8edb736a903c" (UID: "a37f0676-70db-40df-a028-8edb736a903c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:42:10.004815 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:10.004783    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a37f0676-70db-40df-a028-8edb736a903c-kube-api-access-2c4t8" pod "a37f0676-70db-40df-a028-8edb736a903c" (UID: "a37f0676-70db-40df-a028-8edb736a903c"). InnerVolumeSpecName "kube-api-access-2c4t8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:42:10.005327 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:10.005304    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a37f0676-70db-40df-a028-8edb736a903c-maas-api-xks-ca" pod "a37f0676-70db-40df-a028-8edb736a903c" (UID: "a37f0676-70db-40df-a028-8edb736a903c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:42:10.163938 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:10.163889    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a37f0676-70db-40df-a028-8edb736a903c-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:42:10.163938 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:10.163925    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a37f0676-70db-40df-a028-8edb736a903c-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:42:10.163938 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:10.163940    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2c4t8\" (UniqueName: \"kubernetes.io/projected/a37f0676-70db-40df-a028-8edb736a903c-kube-api-access-2c4t8\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:42:10.539898 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:10.535030    5907 patch_prober.go:29] interesting pod/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused" start-of-body=
Oct 30 19:42:10.539898 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:10.535103    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8" podUID="0435effb-f3d6-450f-beed-937cec7badb0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused"
Oct 30 19:42:10.604055 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:10.604030    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-43eebade-tlhqf"
Oct 30 19:42:10.604428 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:10.604098    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-43eebade-tlhqf" event={"ID":"a37f0676-70db-40df-a028-8edb736a903c","Type":"ContainerDied","Data":"e06bb93467f917dd9497ce7ef392ae89a56c322774ee81a51f6572b9060f6d4a"}
Oct 30 19:42:10.604428 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:10.604148    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e06bb93467f917dd9497ce7ef392ae89a56c322774ee81a51f6572b9060f6d4a"
Oct 30 19:42:10.741502 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:10.741404    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"
Oct 30 19:42:10.755855 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:10.755827    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"
Oct 30 19:42:11.545886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:11.544274    5907 patch_prober.go:29] interesting pod/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused" start-of-body=
Oct 30 19:42:11.545886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:11.544332    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8" podUID="0435effb-f3d6-450f-beed-937cec7badb0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused"
Oct 30 19:42:11.695890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:11.689539    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-291d35-7fd55c44f6-fc8sp"]
Oct 30 19:42:11.695890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:11.690976    5907 cpu_manager.go:414] "containerMap: removing container" podUID="a37f0676-70db-40df-a028-8edb736a903c" containerName="revoke-keys"
Oct 30 19:42:11.695890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:11.690997    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a37f0676-70db-40df-a028-8edb736a903c" containerName="revoke-keys"
Oct 30 19:42:11.695890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:11.691288    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="a37f0676-70db-40df-a028-8edb736a903c" containerName="revoke-keys"
Oct 30 19:42:11.698095 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:11.698068    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-291d35-7fd55c44f6-fc8sp"
Oct 30 19:42:11.702761 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:11.702732    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-291d35-7fd55c44f6-fc8sp"]
Oct 30 19:42:11.780338 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:11.780302    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w1-291d35\""
Oct 30 19:42:11.780667 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:11.780645    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w1-291d35-dockercfg-rs8mt\""
Oct 30 19:42:11.826420 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:11.826318    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-291d35-5d99595567-6sjwx"]
Oct 30 19:42:11.911293 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:11.911255    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79819aff-c2ad-43e2-9155-aa91c9c38aa7-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-291d35-7fd55c44f6-fc8sp\" (UID: \"79819aff-c2ad-43e2-9155-aa91c9c38aa7\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-291d35-7fd55c44f6-fc8sp"
Oct 30 19:42:11.911638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:11.911615    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4658q\" (UniqueName: \"kubernetes.io/projected/79819aff-c2ad-43e2-9155-aa91c9c38aa7-kube-api-access-4658q\") pod \"payload-processing-e2e-shared-b-w1-291d35-7fd55c44f6-fc8sp\" (UID: \"79819aff-c2ad-43e2-9155-aa91c9c38aa7\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-291d35-7fd55c44f6-fc8sp"
Oct 30 19:42:12.049755 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:12.049716    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"
Oct 30 19:42:12.050133 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:12.050103    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-291d35-5d99595567-6sjwx"
Oct 30 19:42:12.153300 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:12.121070    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79819aff-c2ad-43e2-9155-aa91c9c38aa7-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-291d35-7fd55c44f6-fc8sp\" (UID: \"79819aff-c2ad-43e2-9155-aa91c9c38aa7\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-291d35-7fd55c44f6-fc8sp"
Oct 30 19:42:12.209086 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:12.123160    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79819aff-c2ad-43e2-9155-aa91c9c38aa7-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-291d35-7fd55c44f6-fc8sp\" (UID: \"79819aff-c2ad-43e2-9155-aa91c9c38aa7\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-291d35-7fd55c44f6-fc8sp"
Oct 30 19:42:12.323718 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:12.310854    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-291d35-5d99595567-6sjwx"]
Oct 30 19:42:12.442955 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:12.389505    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4658q\" (UniqueName: \"kubernetes.io/projected/79819aff-c2ad-43e2-9155-aa91c9c38aa7-kube-api-access-4658q\") pod \"payload-processing-e2e-shared-b-w1-291d35-7fd55c44f6-fc8sp\" (UID: \"79819aff-c2ad-43e2-9155-aa91c9c38aa7\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-291d35-7fd55c44f6-fc8sp"
Oct 30 19:42:12.513208 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:12.490762    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4658q\" (UniqueName: \"kubernetes.io/projected/79819aff-c2ad-43e2-9155-aa91c9c38aa7-kube-api-access-4658q\") pod \"payload-processing-e2e-shared-b-w1-291d35-7fd55c44f6-fc8sp\" (UID: \"79819aff-c2ad-43e2-9155-aa91c9c38aa7\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-291d35-7fd55c44f6-fc8sp"
Oct 30 19:42:12.552662 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:12.552621    5907 patch_prober.go:29] interesting pod/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused" start-of-body=
Oct 30 19:42:12.552981 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:12.552930    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8" podUID="0435effb-f3d6-450f-beed-937cec7badb0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused"
Oct 30 19:42:12.762136 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:12.762030    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zb868\" (UniqueName: \"kubernetes.io/projected/f67cd235-0177-4e87-b233-1e48dd296e3b-kube-api-access-zb868\") pod \"payload-pre-processing-e2e-shared-b-w1-291d35-5d99595567-6sjwx\" (UID: \"f67cd235-0177-4e87-b233-1e48dd296e3b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-291d35-5d99595567-6sjwx"
Oct 30 19:42:12.762669 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:12.762292    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f67cd235-0177-4e87-b233-1e48dd296e3b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-291d35-5d99595567-6sjwx\" (UID: \"f67cd235-0177-4e87-b233-1e48dd296e3b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-291d35-5d99595567-6sjwx"
Oct 30 19:42:12.807194 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:12.807167    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-291d35-7fd55c44f6-fc8sp"
Oct 30 19:42:12.871461 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:12.871419    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f67cd235-0177-4e87-b233-1e48dd296e3b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-291d35-5d99595567-6sjwx\" (UID: \"f67cd235-0177-4e87-b233-1e48dd296e3b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-291d35-5d99595567-6sjwx"
Oct 30 19:42:12.872062 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:12.872034    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zb868\" (UniqueName: \"kubernetes.io/projected/f67cd235-0177-4e87-b233-1e48dd296e3b-kube-api-access-zb868\") pod \"payload-pre-processing-e2e-shared-b-w1-291d35-5d99595567-6sjwx\" (UID: \"f67cd235-0177-4e87-b233-1e48dd296e3b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-291d35-5d99595567-6sjwx"
Oct 30 19:42:12.877396 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:12.877370    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f67cd235-0177-4e87-b233-1e48dd296e3b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-291d35-5d99595567-6sjwx\" (UID: \"f67cd235-0177-4e87-b233-1e48dd296e3b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-291d35-5d99595567-6sjwx"
Oct 30 19:42:12.895431 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:12.895397    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zb868\" (UniqueName: \"kubernetes.io/projected/f67cd235-0177-4e87-b233-1e48dd296e3b-kube-api-access-zb868\") pod \"payload-pre-processing-e2e-shared-b-w1-291d35-5d99595567-6sjwx\" (UID: \"f67cd235-0177-4e87-b233-1e48dd296e3b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-291d35-5d99595567-6sjwx"
Oct 30 19:42:13.180891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:13.178214    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-291d35-5d99595567-6sjwx"
Oct 30 19:42:13.567197 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:13.567093    5907 patch_prober.go:29] interesting pod/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused" start-of-body=
Oct 30 19:42:13.567197 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:13.567160    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8" podUID="0435effb-f3d6-450f-beed-937cec7badb0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused"
Oct 30 19:42:13.666828 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:13.666055    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-291d35-5d99595567-6sjwx"]
Oct 30 19:42:13.670913 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:13.670887    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-291d35-7fd55c44f6-fc8sp"]
Oct 30 19:42:13.671822 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:42:13.671552    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf67cd235_0177_4e87_b233_1e48dd296e3b.slice/crio-e4efae4c8c75ecb220a785758d25a546c22e73cb339fce961160334b9ff67b33 WatchSource:0}: Error finding container e4efae4c8c75ecb220a785758d25a546c22e73cb339fce961160334b9ff67b33: Status 404 returned error can't find the container with id e4efae4c8c75ecb220a785758d25a546c22e73cb339fce961160334b9ff67b33
Oct 30 19:42:13.785875 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:13.785828    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-291d35-5d99595567-6sjwx" event={"ID":"f67cd235-0177-4e87-b233-1e48dd296e3b","Type":"ContainerStarted","Data":"e4efae4c8c75ecb220a785758d25a546c22e73cb339fce961160334b9ff67b33"}
Oct 30 19:42:13.787301 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:13.787276    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-291d35-7fd55c44f6-fc8sp" event={"ID":"79819aff-c2ad-43e2-9155-aa91c9c38aa7","Type":"ContainerStarted","Data":"98c78f6243b608422ded8e29caae7e96a443138b627a6e81063e9c0ac61dfb86"}
Oct 30 19:42:14.540152 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:14.540121    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:42:14.541513 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:14.541489    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:42:14.982679 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:14.982022    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-291d35-5d99595567-6sjwx" event={"ID":"f67cd235-0177-4e87-b233-1e48dd296e3b","Type":"ContainerStarted","Data":"538d929616656ff4f5b6c8ce2500a8ef7da03a37efdc82fb1dc5b2976269f2af"}
Oct 30 19:42:14.982679 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:14.982107    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-291d35-5d99595567-6sjwx"
Oct 30 19:42:14.985166 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:14.985134    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-291d35-7fd55c44f6-fc8sp" event={"ID":"79819aff-c2ad-43e2-9155-aa91c9c38aa7","Type":"ContainerStarted","Data":"d73ac53c62eb070bea0bb17c082707f54e328208ed51295b60beece727360530"}
Oct 30 19:42:14.985402 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:14.985388    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-291d35-7fd55c44f6-fc8sp"
Oct 30 19:42:15.059852 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:15.059799    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-291d35-5d99595567-6sjwx" podStartSLOduration=4.059784997 podStartE2EDuration="4.059784997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:42:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:42:15.056925162 +0000 UTC m=+1246.899857101" watchObservedRunningTime="2026-10-30 19:42:15.059784997 +0000 UTC m=+1246.902716938"
Oct 30 19:42:15.138390 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:15.138323    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-291d35-7fd55c44f6-fc8sp" podStartSLOduration=4.138303716 podStartE2EDuration="4.138303716s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:42:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:42:15.136046555 +0000 UTC m=+1246.978978494" watchObservedRunningTime="2026-10-30 19:42:15.138303716 +0000 UTC m=+1246.981235664"
Oct 30 19:42:17.505120 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:42:17.505057    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:42:21.188898 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:21.188825    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-34bdc0-kjwss"]
Oct 30 19:42:21.193999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:21.193977    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-34bdc0-kjwss"
Oct 30 19:42:21.218506 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:21.218427    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-34bdc0-kjwss"]
Oct 30 19:42:21.224197 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:21.224158    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:42:21.224395 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:21.224265    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:42:21.354299 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:21.354259    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8rsv\" (UniqueName: \"kubernetes.io/projected/07e7453d-1591-404e-98ff-6be69a652bd3-kube-api-access-l8rsv\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-34bdc0-kjwss\" (UID: \"07e7453d-1591-404e-98ff-6be69a652bd3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-34bdc0-kjwss"
Oct 30 19:42:21.354493 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:21.354456    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/07e7453d-1591-404e-98ff-6be69a652bd3-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-34bdc0-kjwss\" (UID: \"07e7453d-1591-404e-98ff-6be69a652bd3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-34bdc0-kjwss"
Oct 30 19:42:21.354726 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:21.354679    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/07e7453d-1591-404e-98ff-6be69a652bd3-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-34bdc0-kjwss\" (UID: \"07e7453d-1591-404e-98ff-6be69a652bd3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-34bdc0-kjwss"
Oct 30 19:42:21.456267 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:21.456183    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8rsv\" (UniqueName: \"kubernetes.io/projected/07e7453d-1591-404e-98ff-6be69a652bd3-kube-api-access-l8rsv\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-34bdc0-kjwss\" (UID: \"07e7453d-1591-404e-98ff-6be69a652bd3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-34bdc0-kjwss"
Oct 30 19:42:21.456447 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:21.456333    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/07e7453d-1591-404e-98ff-6be69a652bd3-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-34bdc0-kjwss\" (UID: \"07e7453d-1591-404e-98ff-6be69a652bd3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-34bdc0-kjwss"
Oct 30 19:42:21.456527 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:21.456507    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/07e7453d-1591-404e-98ff-6be69a652bd3-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-34bdc0-kjwss\" (UID: \"07e7453d-1591-404e-98ff-6be69a652bd3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-34bdc0-kjwss"
Oct 30 19:42:21.457403 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:21.457373    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/07e7453d-1591-404e-98ff-6be69a652bd3-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-34bdc0-kjwss\" (UID: \"07e7453d-1591-404e-98ff-6be69a652bd3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-34bdc0-kjwss"
Oct 30 19:42:21.459819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:21.459790    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/07e7453d-1591-404e-98ff-6be69a652bd3-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-34bdc0-kjwss\" (UID: \"07e7453d-1591-404e-98ff-6be69a652bd3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-34bdc0-kjwss"
Oct 30 19:42:21.478321 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:21.478291    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8rsv\" (UniqueName: \"kubernetes.io/projected/07e7453d-1591-404e-98ff-6be69a652bd3-kube-api-access-l8rsv\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-34bdc0-kjwss\" (UID: \"07e7453d-1591-404e-98ff-6be69a652bd3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-34bdc0-kjwss"
Oct 30 19:42:21.511252 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:21.511214    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-34bdc0-kjwss"
Oct 30 19:42:21.888755 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:21.888705    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-34bdc0-kjwss"]
Oct 30 19:42:21.899335 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:42:21.899307    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07e7453d_1591_404e_98ff_6be69a652bd3.slice/crio-7f0edbf8217f297c1ca75e4256e34920b1cae176a739ef206876cefe07a8879f WatchSource:0}: Error finding container 7f0edbf8217f297c1ca75e4256e34920b1cae176a739ef206876cefe07a8879f: Status 404 returned error can't find the container with id 7f0edbf8217f297c1ca75e4256e34920b1cae176a739ef206876cefe07a8879f
Oct 30 19:42:22.071339 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:22.071301    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-34bdc0-kjwss" event={"ID":"07e7453d-1591-404e-98ff-6be69a652bd3","Type":"ContainerStarted","Data":"7f0edbf8217f297c1ca75e4256e34920b1cae176a739ef206876cefe07a8879f"}
Oct 30 19:42:23.083015 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:23.082982    5907 generic.go:356] "Generic (PLEG): container finished" podID="29fbb7be-9068-40ec-a0d4-9fa0ed0469bf" containerID="e3a35def6621ba550ad37eeb67feec3899966bcf15d06ef4badc081f7d5849eb" exitCode=0
Oct 30 19:42:23.083514 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:23.083059    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp" event={"ID":"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf","Type":"ContainerDied","Data":"e3a35def6621ba550ad37eeb67feec3899966bcf15d06ef4badc081f7d5849eb"}
Oct 30 19:42:23.083703 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:23.083686    5907 scope.go:122] "RemoveContainer" containerID="e3a35def6621ba550ad37eeb67feec3899966bcf15d06ef4badc081f7d5849eb"
Oct 30 19:42:23.085166 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:23.085143    5907 generic.go:356] "Generic (PLEG): container finished" podID="07e7453d-1591-404e-98ff-6be69a652bd3" containerID="e70e17cf0457faea617aa7028cd2877bcf5f1e85f72bf87fa68fb5f3279b91ef" exitCode=0
Oct 30 19:42:23.085293 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:23.085177    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-34bdc0-kjwss" event={"ID":"07e7453d-1591-404e-98ff-6be69a652bd3","Type":"ContainerDied","Data":"e70e17cf0457faea617aa7028cd2877bcf5f1e85f72bf87fa68fb5f3279b91ef"}
Oct 30 19:42:23.085740 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:23.085709    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:42:23.085838 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:23.085799    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:42:23.085952 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:23.085852    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:42:24.036914 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:24.036884    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e3b90b28-28lz8"]
Oct 30 19:42:24.050189 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:24.050162    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e3b90b28-28lz8"]
Oct 30 19:42:24.096842 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:24.096803    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp" event={"ID":"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf","Type":"ContainerStarted","Data":"7d3b1f55131c855d23bf3ff42421397abc230221ce98296c182f1e26ace045d6"}
Oct 30 19:42:24.286350 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:24.286322    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-34bdc0-kjwss"
Oct 30 19:42:24.292113 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:24.292052    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1c7e341b-211b-4628-a7f6-e96271655434" path="/var/lib/kubelet/pods/1c7e341b-211b-4628-a7f6-e96271655434/volumes"
Oct 30 19:42:24.337341 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:24.337312    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/07e7453d-1591-404e-98ff-6be69a652bd3-kube-api-access-l8rsv\" (UniqueName: \"kubernetes.io/projected/07e7453d-1591-404e-98ff-6be69a652bd3-kube-api-access-l8rsv\") pod \"07e7453d-1591-404e-98ff-6be69a652bd3\" (UID: \"07e7453d-1591-404e-98ff-6be69a652bd3\") "
Oct 30 19:42:24.337657 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:24.337638    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/07e7453d-1591-404e-98ff-6be69a652bd3-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/07e7453d-1591-404e-98ff-6be69a652bd3-maas-api-xks-ca\") pod \"07e7453d-1591-404e-98ff-6be69a652bd3\" (UID: \"07e7453d-1591-404e-98ff-6be69a652bd3\") "
Oct 30 19:42:24.338216 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:24.337721    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/07e7453d-1591-404e-98ff-6be69a652bd3-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/07e7453d-1591-404e-98ff-6be69a652bd3-maas-api-openshift-ca\") pod \"07e7453d-1591-404e-98ff-6be69a652bd3\" (UID: \"07e7453d-1591-404e-98ff-6be69a652bd3\") "
Oct 30 19:42:24.339949 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:24.339853    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07e7453d-1591-404e-98ff-6be69a652bd3-maas-api-openshift-ca" pod "07e7453d-1591-404e-98ff-6be69a652bd3" (UID: "07e7453d-1591-404e-98ff-6be69a652bd3"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:42:24.340910 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:24.340845    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07e7453d-1591-404e-98ff-6be69a652bd3-kube-api-access-l8rsv" pod "07e7453d-1591-404e-98ff-6be69a652bd3" (UID: "07e7453d-1591-404e-98ff-6be69a652bd3"). InnerVolumeSpecName "kube-api-access-l8rsv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:42:24.341344 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:24.341320    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07e7453d-1591-404e-98ff-6be69a652bd3-maas-api-xks-ca" pod "07e7453d-1591-404e-98ff-6be69a652bd3" (UID: "07e7453d-1591-404e-98ff-6be69a652bd3"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:42:24.440745 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:24.440710    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8rsv\" (UniqueName: \"kubernetes.io/projected/07e7453d-1591-404e-98ff-6be69a652bd3-kube-api-access-l8rsv\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:42:24.440745 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:24.440741    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/07e7453d-1591-404e-98ff-6be69a652bd3-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:42:24.440981 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:24.440760    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/07e7453d-1591-404e-98ff-6be69a652bd3-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:42:24.448108 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:24.448081    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:42:24.449431 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:24.449405    5907 patch_prober.go:29] interesting pod/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.172:15021/healthz/ready\": dial tcp 10.134.0.172:15021: connect: connection refused" start-of-body=
Oct 30 19:42:24.449506 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:24.449456    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp" podUID="29fbb7be-9068-40ec-a0d4-9fa0ed0469bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.172:15021/healthz/ready\": dial tcp 10.134.0.172:15021: connect: connection refused"
Oct 30 19:42:25.107557 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:25.107527    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-34bdc0-kjwss"
Oct 30 19:42:25.108066 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:25.107526    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-34bdc0-kjwss" event={"ID":"07e7453d-1591-404e-98ff-6be69a652bd3","Type":"ContainerDied","Data":"7f0edbf8217f297c1ca75e4256e34920b1cae176a739ef206876cefe07a8879f"}
Oct 30 19:42:25.108066 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:25.107662    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7f0edbf8217f297c1ca75e4256e34920b1cae176a739ef206876cefe07a8879f"
Oct 30 19:42:25.335075 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:25.335032    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:42:25.447430 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:25.447397    5907 patch_prober.go:29] interesting pod/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.172:15021/healthz/ready\": dial tcp 10.134.0.172:15021: connect: connection refused" start-of-body=
Oct 30 19:42:25.447627 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:25.447461    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp" podUID="29fbb7be-9068-40ec-a0d4-9fa0ed0469bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.172:15021/healthz/ready\": dial tcp 10.134.0.172:15021: connect: connection refused"
Oct 30 19:42:26.013888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:26.013054    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-291d35-5d99595567-6sjwx"
Oct 30 19:42:26.052894 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:26.051022    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-291d35-7fd55c44f6-fc8sp"
Oct 30 19:42:26.447666 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:26.447625    5907 patch_prober.go:29] interesting pod/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.172:15021/healthz/ready\": dial tcp 10.134.0.172:15021: connect: connection refused" start-of-body=
Oct 30 19:42:26.448133 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:26.447689    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp" podUID="29fbb7be-9068-40ec-a0d4-9fa0ed0469bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.172:15021/healthz/ready\": dial tcp 10.134.0.172:15021: connect: connection refused"
Oct 30 19:42:27.448044 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:27.448007    5907 patch_prober.go:29] interesting pod/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.172:15021/healthz/ready\": dial tcp 10.134.0.172:15021: connect: connection refused" start-of-body=
Oct 30 19:42:27.448518 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:27.448069    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp" podUID="29fbb7be-9068-40ec-a0d4-9fa0ed0469bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.172:15021/healthz/ready\": dial tcp 10.134.0.172:15021: connect: connection refused"
Oct 30 19:42:28.459035 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:28.452341    5907 patch_prober.go:29] interesting pod/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.172:15021/healthz/ready\": dial tcp 10.134.0.172:15021: connect: connection refused" start-of-body=
Oct 30 19:42:28.459035 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:28.452401    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp" podUID="29fbb7be-9068-40ec-a0d4-9fa0ed0469bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.172:15021/healthz/ready\": dial tcp 10.134.0.172:15021: connect: connection refused"
Oct 30 19:42:29.448365 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:29.448318    5907 patch_prober.go:29] interesting pod/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.172:15021/healthz/ready\": dial tcp 10.134.0.172:15021: connect: connection refused" start-of-body=
Oct 30 19:42:29.448596 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:29.448387    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp" podUID="29fbb7be-9068-40ec-a0d4-9fa0ed0469bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.172:15021/healthz/ready\": dial tcp 10.134.0.172:15021: connect: connection refused"
Oct 30 19:42:29.863941 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:29.855499    5907 scope.go:122] "RemoveContainer" containerID="8597a94fb771373d034e10b64022f3b5e5cac5e9e7184a779ee69d25a2d82310"
Oct 30 19:42:29.936134 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:29.930113    5907 scope.go:122] "RemoveContainer" containerID="8a74ec580f59fd9f09d0d4acb07dae7f4a2ea2b4e45bc7de3cd05143486aefca"
Oct 30 19:42:29.958735 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:29.958695    5907 scope.go:122] "RemoveContainer" containerID="9e46ccb6e53e87ccc3262f30109ed8074ee0d6eda3297100fec31b09d089f8ed"
Oct 30 19:42:29.969954 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:29.969930    5907 scope.go:122] "RemoveContainer" containerID="7242303407d5a55d80f9b1cd760133fd57b2037a3ad30ea19810a6cc26c561fd"
Oct 30 19:42:29.980285 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:29.980261    5907 scope.go:122] "RemoveContainer" containerID="d116edb9fac502126ee14e02503d8990590c55ef2d15607a55efa3d9b03fd7a0"
Oct 30 19:42:29.990589 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:29.990564    5907 scope.go:122] "RemoveContainer" containerID="ebf8ea7c835cad660ab905ef1cefae1808336d7b9404b10488b854243f7c8ac2"
Oct 30 19:42:30.002032 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:30.002009    5907 scope.go:122] "RemoveContainer" containerID="b0a961f12815accaeb2507f32ef6cafaff6d25bb57f48a4a2c84f400789e4389"
Oct 30 19:42:30.037963 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:30.037936    5907 scope.go:122] "RemoveContainer" containerID="e8a3f7be371f4c05acd917a6d2a515e94b4d97118bf9c120364bf041d29e8463"
Oct 30 19:42:30.052507 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:30.051966    5907 scope.go:122] "RemoveContainer" containerID="be564d018ebd236e70116021c8c401acc400a2b5974b30a689c0ae1a56573b5c"
Oct 30 19:42:30.064146 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:30.064119    5907 scope.go:122] "RemoveContainer" containerID="6e141defb52daa82e1325d196f68c7cbbe8ae957dcf67b6efa551375a15e98c7"
Oct 30 19:42:30.074596 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:30.074575    5907 scope.go:122] "RemoveContainer" containerID="c0dc170c9b8029cd686672e625d5cb511772c7e67a4e838ea5e8e28f43f124f6"
Oct 30 19:42:30.084669 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:30.084648    5907 scope.go:122] "RemoveContainer" containerID="482d8ba069499aee81689ba6fb159c0b5ce5ae88ec9a11600dc545d91d0cf4f1"
Oct 30 19:42:30.453080 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:30.452997    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:42:30.454047 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:30.454027    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:42:32.359394 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:32.359359    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-34bdc0-56c759c9d-dch72"]
Oct 30 19:42:32.394209 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:32.394166    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-34bdc0-6c45d6ff5-dxrqw"]
Oct 30 19:42:32.535069 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:32.535038    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:42:37.367889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:37.367226    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-34bdc0-56c759c9d-dch72" podUID="ee095b66-4a1d-491c-9e6d-3a189a80830a" containerName="payload-processing" containerID="cri-o://fd8c7f4266825f99110ff0d3138c949ae8f053570c5a4614e57d4aceda65b483" gracePeriod=25
Oct 30 19:42:37.396187 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:37.396092    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-34bdc0-6c45d6ff5-dxrqw" podUID="6b44cab3-caa0-42ac-a40e-6f8ff8008c61" containerName="payload-pre-processing" containerID="cri-o://81a529764aac4a94621e2550564953070b87e90f5151f11fa060a91a7e95fc78" gracePeriod=25
Oct 30 19:42:38.107065 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.069830    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:42:38.369904 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.369873    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-34bdc0-56c759c9d-dch72"
Oct 30 19:42:38.441396 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.441365    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-34bdc0-6c45d6ff5-dxrqw"
Oct 30 19:42:38.471073 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.471043    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ee095b66-4a1d-491c-9e6d-3a189a80830a-kube-api-access-7vxd4\" (UniqueName: \"kubernetes.io/projected/ee095b66-4a1d-491c-9e6d-3a189a80830a-kube-api-access-7vxd4\") pod \"ee095b66-4a1d-491c-9e6d-3a189a80830a\" (UID: \"ee095b66-4a1d-491c-9e6d-3a189a80830a\") "
Oct 30 19:42:38.471254 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.471162    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ee095b66-4a1d-491c-9e6d-3a189a80830a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee095b66-4a1d-491c-9e6d-3a189a80830a-plugins-config-volume\") pod \"ee095b66-4a1d-491c-9e6d-3a189a80830a\" (UID: \"ee095b66-4a1d-491c-9e6d-3a189a80830a\") "
Oct 30 19:42:38.471401 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.471380    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6b44cab3-caa0-42ac-a40e-6f8ff8008c61-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b44cab3-caa0-42ac-a40e-6f8ff8008c61-plugins-config-volume\") pod \"6b44cab3-caa0-42ac-a40e-6f8ff8008c61\" (UID: \"6b44cab3-caa0-42ac-a40e-6f8ff8008c61\") "
Oct 30 19:42:38.471485 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.471419    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6b44cab3-caa0-42ac-a40e-6f8ff8008c61-kube-api-access-9n26x\" (UniqueName: \"kubernetes.io/projected/6b44cab3-caa0-42ac-a40e-6f8ff8008c61-kube-api-access-9n26x\") pod \"6b44cab3-caa0-42ac-a40e-6f8ff8008c61\" (UID: \"6b44cab3-caa0-42ac-a40e-6f8ff8008c61\") "
Oct 30 19:42:38.471746 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.471608    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee095b66-4a1d-491c-9e6d-3a189a80830a-plugins-config-volume" pod "ee095b66-4a1d-491c-9e6d-3a189a80830a" (UID: "ee095b66-4a1d-491c-9e6d-3a189a80830a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:42:38.471837 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.471807    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b44cab3-caa0-42ac-a40e-6f8ff8008c61-plugins-config-volume" pod "6b44cab3-caa0-42ac-a40e-6f8ff8008c61" (UID: "6b44cab3-caa0-42ac-a40e-6f8ff8008c61"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:42:38.472421 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.472397    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee095b66-4a1d-491c-9e6d-3a189a80830a-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:42:38.472529 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.472428    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b44cab3-caa0-42ac-a40e-6f8ff8008c61-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:42:38.474400 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.474373    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b44cab3-caa0-42ac-a40e-6f8ff8008c61-kube-api-access-9n26x" pod "6b44cab3-caa0-42ac-a40e-6f8ff8008c61" (UID: "6b44cab3-caa0-42ac-a40e-6f8ff8008c61"). InnerVolumeSpecName "kube-api-access-9n26x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:42:38.479617 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.479559    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee095b66-4a1d-491c-9e6d-3a189a80830a-kube-api-access-7vxd4" pod "ee095b66-4a1d-491c-9e6d-3a189a80830a" (UID: "ee095b66-4a1d-491c-9e6d-3a189a80830a"). InnerVolumeSpecName "kube-api-access-7vxd4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:42:38.540957 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.540914    5907 generic.go:356] "Generic (PLEG): container finished" podID="6b44cab3-caa0-42ac-a40e-6f8ff8008c61" containerID="81a529764aac4a94621e2550564953070b87e90f5151f11fa060a91a7e95fc78" exitCode=0
Oct 30 19:42:38.540957 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.540956    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-34bdc0-6c45d6ff5-dxrqw"
Oct 30 19:42:38.541212 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.540996    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-34bdc0-6c45d6ff5-dxrqw" event={"ID":"6b44cab3-caa0-42ac-a40e-6f8ff8008c61","Type":"ContainerDied","Data":"81a529764aac4a94621e2550564953070b87e90f5151f11fa060a91a7e95fc78"}
Oct 30 19:42:38.541212 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.541057    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-34bdc0-6c45d6ff5-dxrqw" event={"ID":"6b44cab3-caa0-42ac-a40e-6f8ff8008c61","Type":"ContainerDied","Data":"d79282aac46abee980fdf5c7101495bce53f2ff9a57d80dd7276d283ca901fc5"}
Oct 30 19:42:38.541212 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.541092    5907 scope.go:122] "RemoveContainer" containerID="81a529764aac4a94621e2550564953070b87e90f5151f11fa060a91a7e95fc78"
Oct 30 19:42:38.543168 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.542960    5907 generic.go:356] "Generic (PLEG): container finished" podID="ee095b66-4a1d-491c-9e6d-3a189a80830a" containerID="fd8c7f4266825f99110ff0d3138c949ae8f053570c5a4614e57d4aceda65b483" exitCode=0
Oct 30 19:42:38.543168 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.543041    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-34bdc0-56c759c9d-dch72"
Oct 30 19:42:38.543168 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.543085    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-34bdc0-56c759c9d-dch72" event={"ID":"ee095b66-4a1d-491c-9e6d-3a189a80830a","Type":"ContainerDied","Data":"fd8c7f4266825f99110ff0d3138c949ae8f053570c5a4614e57d4aceda65b483"}
Oct 30 19:42:38.543168 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.543130    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-34bdc0-56c759c9d-dch72" event={"ID":"ee095b66-4a1d-491c-9e6d-3a189a80830a","Type":"ContainerDied","Data":"5acbf630e17c651986db789205d98cfbf1bd2232d3bae7bf54f77e1956d0ee1b"}
Oct 30 19:42:38.608655 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.606612    5907 scope.go:122] "RemoveContainer" containerID="81a529764aac4a94621e2550564953070b87e90f5151f11fa060a91a7e95fc78"
Oct 30 19:42:38.616385 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:42:38.616157    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81a529764aac4a94621e2550564953070b87e90f5151f11fa060a91a7e95fc78\": container with ID starting with 81a529764aac4a94621e2550564953070b87e90f5151f11fa060a91a7e95fc78 not found: ID does not exist" containerID="81a529764aac4a94621e2550564953070b87e90f5151f11fa060a91a7e95fc78"
Oct 30 19:42:38.616385 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.616208    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81a529764aac4a94621e2550564953070b87e90f5151f11fa060a91a7e95fc78"} err="failed to get container status \"81a529764aac4a94621e2550564953070b87e90f5151f11fa060a91a7e95fc78\": rpc error: code = NotFound desc = could not find container \"81a529764aac4a94621e2550564953070b87e90f5151f11fa060a91a7e95fc78\": container with ID starting with 81a529764aac4a94621e2550564953070b87e90f5151f11fa060a91a7e95fc78 not found: ID does not exist"
Oct 30 19:42:38.616385 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.616235    5907 scope.go:122] "RemoveContainer" containerID="fd8c7f4266825f99110ff0d3138c949ae8f053570c5a4614e57d4aceda65b483"
Oct 30 19:42:38.631402 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.631282    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9n26x\" (UniqueName: \"kubernetes.io/projected/6b44cab3-caa0-42ac-a40e-6f8ff8008c61-kube-api-access-9n26x\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:42:38.631402 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.631368    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vxd4\" (UniqueName: \"kubernetes.io/projected/ee095b66-4a1d-491c-9e6d-3a189a80830a-kube-api-access-7vxd4\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:42:38.672763 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.662080    5907 scope.go:122] "RemoveContainer" containerID="fd8c7f4266825f99110ff0d3138c949ae8f053570c5a4614e57d4aceda65b483"
Oct 30 19:42:38.675405 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:42:38.675285    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd8c7f4266825f99110ff0d3138c949ae8f053570c5a4614e57d4aceda65b483\": container with ID starting with fd8c7f4266825f99110ff0d3138c949ae8f053570c5a4614e57d4aceda65b483 not found: ID does not exist" containerID="fd8c7f4266825f99110ff0d3138c949ae8f053570c5a4614e57d4aceda65b483"
Oct 30 19:42:38.675405 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.675334    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd8c7f4266825f99110ff0d3138c949ae8f053570c5a4614e57d4aceda65b483"} err="failed to get container status \"fd8c7f4266825f99110ff0d3138c949ae8f053570c5a4614e57d4aceda65b483\": rpc error: code = NotFound desc = could not find container \"fd8c7f4266825f99110ff0d3138c949ae8f053570c5a4614e57d4aceda65b483\": container with ID starting with fd8c7f4266825f99110ff0d3138c949ae8f053570c5a4614e57d4aceda65b483 not found: ID does not exist"
Oct 30 19:42:38.684940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.683960    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-34bdc0-6c45d6ff5-dxrqw"]
Oct 30 19:42:38.689750 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.689718    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-34bdc0-6c45d6ff5-dxrqw"]
Oct 30 19:42:38.701220 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.701184    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-34bdc0-56c759c9d-dch72"]
Oct 30 19:42:38.706155 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.706127    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-34bdc0-56c759c9d-dch72"]
Oct 30 19:42:38.826747 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.826659    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"]
Oct 30 19:42:38.833130 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.833099    5907 cpu_manager.go:414] "containerMap: removing container" podUID="07e7453d-1591-404e-98ff-6be69a652bd3" containerName="revoke-keys"
Oct 30 19:42:38.833130 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.833128    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="07e7453d-1591-404e-98ff-6be69a652bd3" containerName="revoke-keys"
Oct 30 19:42:38.833322 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.833161    5907 cpu_manager.go:414] "containerMap: removing container" podUID="ee095b66-4a1d-491c-9e6d-3a189a80830a" containerName="payload-processing"
Oct 30 19:42:38.833322 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.833169    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ee095b66-4a1d-491c-9e6d-3a189a80830a" containerName="payload-processing"
Oct 30 19:42:38.833322 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.833250    5907 cpu_manager.go:414] "containerMap: removing container" podUID="6b44cab3-caa0-42ac-a40e-6f8ff8008c61" containerName="payload-pre-processing"
Oct 30 19:42:38.833322 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.833258    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6b44cab3-caa0-42ac-a40e-6f8ff8008c61" containerName="payload-pre-processing"
Oct 30 19:42:38.833498 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.833480    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="07e7453d-1591-404e-98ff-6be69a652bd3" containerName="revoke-keys"
Oct 30 19:42:38.833541 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.833506    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="ee095b66-4a1d-491c-9e6d-3a189a80830a" containerName="payload-processing"
Oct 30 19:42:38.833541 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.833534    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="6b44cab3-caa0-42ac-a40e-6f8ff8008c61" containerName="payload-pre-processing"
Oct 30 19:42:38.839198 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.839173    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:42:38.856066 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.856039    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"]
Oct 30 19:42:38.867324 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:38.867298    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-086f2a-openshift-default-dockercfg-l5msk\""
Oct 30 19:42:39.025045 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:39.022400    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whs48\" (UniqueName: \"kubernetes.io/projected/320e693e-a39d-4e8b-9772-cc8e811acc69-kube-api-access-whs48\") pod \"e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") " pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:42:39.025045 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:39.022512    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/320e693e-a39d-4e8b-9772-cc8e811acc69-workload-certs\") pod \"e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") " pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:42:39.025045 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:39.022633    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/320e693e-a39d-4e8b-9772-cc8e811acc69-istiod-ca-cert\") pod \"e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") " pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:42:39.025045 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:39.022698    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/320e693e-a39d-4e8b-9772-cc8e811acc69-istio-podinfo\") pod \"e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") " pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:42:39.025045 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:39.022811    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/320e693e-a39d-4e8b-9772-cc8e811acc69-workload-socket\") pod \"e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") " pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:42:39.025045 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:39.022911    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/320e693e-a39d-4e8b-9772-cc8e811acc69-credential-socket\") pod \"e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") " pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:42:39.025045 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:39.022994    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/320e693e-a39d-4e8b-9772-cc8e811acc69-istio-envoy\") pod \"e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") " pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:42:39.025045 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:39.023019    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/320e693e-a39d-4e8b-9772-cc8e811acc69-istio-data\") pod \"e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") " pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:42:39.025045 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:39.023191    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/320e693e-a39d-4e8b-9772-cc8e811acc69-istio-token\") pod \"e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") " pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:42:39.138882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:39.134479    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/320e693e-a39d-4e8b-9772-cc8e811acc69-workload-certs\") pod \"e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") " pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:42:39.138882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:39.134599    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/320e693e-a39d-4e8b-9772-cc8e811acc69-istiod-ca-cert\") pod \"e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") " pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:42:39.138882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:39.134661    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/320e693e-a39d-4e8b-9772-cc8e811acc69-istio-podinfo\") pod \"e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") " pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:42:39.138882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:39.134749    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/320e693e-a39d-4e8b-9772-cc8e811acc69-workload-socket\") pod \"e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") " pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:42:39.138882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:39.134818    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/320e693e-a39d-4e8b-9772-cc8e811acc69-credential-socket\") pod \"e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") " pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:42:39.138882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:39.134907    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/320e693e-a39d-4e8b-9772-cc8e811acc69-istio-envoy\") pod \"e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") " pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:42:39.138882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:39.134930    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/320e693e-a39d-4e8b-9772-cc8e811acc69-istio-data\") pod \"e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") " pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:42:39.138882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:39.135076    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/320e693e-a39d-4e8b-9772-cc8e811acc69-istio-token\") pod \"e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") " pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:42:39.138882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:39.135134    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whs48\" (UniqueName: \"kubernetes.io/projected/320e693e-a39d-4e8b-9772-cc8e811acc69-kube-api-access-whs48\") pod \"e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") " pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:42:39.138882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:39.135953    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/320e693e-a39d-4e8b-9772-cc8e811acc69-workload-certs\") pod \"e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") " pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:42:39.138882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:39.136640    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/320e693e-a39d-4e8b-9772-cc8e811acc69-istiod-ca-cert\") pod \"e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") " pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:42:39.202895 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:39.173268    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/320e693e-a39d-4e8b-9772-cc8e811acc69-workload-socket\") pod \"e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") " pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:42:39.202895 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:39.173487    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/320e693e-a39d-4e8b-9772-cc8e811acc69-credential-socket\") pod \"e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") " pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:42:39.202895 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:39.177259    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/320e693e-a39d-4e8b-9772-cc8e811acc69-istio-data\") pod \"e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") " pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:42:39.311947 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:39.303429    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/320e693e-a39d-4e8b-9772-cc8e811acc69-istio-podinfo\") pod \"e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") " pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:42:39.343777 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:39.343711    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/320e693e-a39d-4e8b-9772-cc8e811acc69-istio-token\") pod \"e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") " pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:42:39.363032 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:39.362840    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/320e693e-a39d-4e8b-9772-cc8e811acc69-istio-envoy\") pod \"e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") " pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:42:39.368089 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:39.368058    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-whs48\" (UniqueName: \"kubernetes.io/projected/320e693e-a39d-4e8b-9772-cc8e811acc69-kube-api-access-whs48\") pod \"e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") " pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:42:39.459559 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:39.459204    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:42:39.966380 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:39.966345    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"]
Oct 30 19:42:40.040808 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:40.040739    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:42:40.041051 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:40.040938    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:42:40.041204 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:40.041056    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:42:40.292120 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:40.292083    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6b44cab3-caa0-42ac-a40e-6f8ff8008c61" path="/var/lib/kubelet/pods/6b44cab3-caa0-42ac-a40e-6f8ff8008c61/volumes"
Oct 30 19:42:40.292640 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:40.292616    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ee095b66-4a1d-491c-9e6d-3a189a80830a" path="/var/lib/kubelet/pods/ee095b66-4a1d-491c-9e6d-3a189a80830a/volumes"
Oct 30 19:42:40.661751 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:40.660068    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4" event={"ID":"320e693e-a39d-4e8b-9772-cc8e811acc69","Type":"ContainerStarted","Data":"12edb43d3409e56c27f3fc6e214c9f1184daea65e1a9bdf5f804d720eb82ece0"}
Oct 30 19:42:40.661751 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:40.660140    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4" event={"ID":"320e693e-a39d-4e8b-9772-cc8e811acc69","Type":"ContainerStarted","Data":"67242aeccfb3501e19b0a4d3a0a5e7d674b07a1d9f2a64faa9b7284fc14d396e"}
Oct 30 19:42:40.726012 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:40.725964    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4" podStartSLOduration=2.725950181 podStartE2EDuration="2.725950181s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:42:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:42:40.722096327 +0000 UTC m=+1272.565028261" watchObservedRunningTime="2026-10-30 19:42:40.725950181 +0000 UTC m=+1272.568882120"
Oct 30 19:42:41.460736 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:41.460705    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:42:41.466897 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:41.466839    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:42:41.716410 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:41.716331    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:42:41.723453 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:41.723421    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:42:44.956693 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:44.956635    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-086f2a-9b5f7cf-z2579"]
Oct 30 19:42:44.962745 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:44.962725    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-086f2a-9b5f7cf-z2579"
Oct 30 19:42:44.985729 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:44.985703    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-086f2a-9b5f7cf-z2579"]
Oct 30 19:42:44.993709 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:44.993683    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-086f2a\""
Oct 30 19:42:44.993709 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:44.993684    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-086f2a-dockercfg-ql8nf\""
Oct 30 19:42:45.141886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:45.133759    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56b5t\" (UniqueName: \"kubernetes.io/projected/d925ef89-ce40-4ed1-ad1a-eadd4778245e-kube-api-access-56b5t\") pod \"payload-pre-processing-e2e-api-b-086f2a-9b5f7cf-z2579\" (UID: \"d925ef89-ce40-4ed1-ad1a-eadd4778245e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-086f2a-9b5f7cf-z2579"
Oct 30 19:42:45.141886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:45.134045    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d925ef89-ce40-4ed1-ad1a-eadd4778245e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-086f2a-9b5f7cf-z2579\" (UID: \"d925ef89-ce40-4ed1-ad1a-eadd4778245e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-086f2a-9b5f7cf-z2579"
Oct 30 19:42:45.255938 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:45.249067    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56b5t\" (UniqueName: \"kubernetes.io/projected/d925ef89-ce40-4ed1-ad1a-eadd4778245e-kube-api-access-56b5t\") pod \"payload-pre-processing-e2e-api-b-086f2a-9b5f7cf-z2579\" (UID: \"d925ef89-ce40-4ed1-ad1a-eadd4778245e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-086f2a-9b5f7cf-z2579"
Oct 30 19:42:45.255938 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:45.249451    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d925ef89-ce40-4ed1-ad1a-eadd4778245e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-086f2a-9b5f7cf-z2579\" (UID: \"d925ef89-ce40-4ed1-ad1a-eadd4778245e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-086f2a-9b5f7cf-z2579"
Oct 30 19:42:45.255938 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:45.250760    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d925ef89-ce40-4ed1-ad1a-eadd4778245e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-086f2a-9b5f7cf-z2579\" (UID: \"d925ef89-ce40-4ed1-ad1a-eadd4778245e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-086f2a-9b5f7cf-z2579"
Oct 30 19:42:45.329462 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:45.329433    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56b5t\" (UniqueName: \"kubernetes.io/projected/d925ef89-ce40-4ed1-ad1a-eadd4778245e-kube-api-access-56b5t\") pod \"payload-pre-processing-e2e-api-b-086f2a-9b5f7cf-z2579\" (UID: \"d925ef89-ce40-4ed1-ad1a-eadd4778245e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-086f2a-9b5f7cf-z2579"
Oct 30 19:42:45.586763 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:45.586662    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-086f2a-9b5f7cf-z2579"
Oct 30 19:42:46.078976 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:42:46.078943    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd925ef89_ce40_4ed1_ad1a_eadd4778245e.slice/crio-e5413a4b665606c39fa9a4abaf5ba4f73788c178d5df26014f7946e29d1ca4dc WatchSource:0}: Error finding container e5413a4b665606c39fa9a4abaf5ba4f73788c178d5df26014f7946e29d1ca4dc: Status 404 returned error can't find the container with id e5413a4b665606c39fa9a4abaf5ba4f73788c178d5df26014f7946e29d1ca4dc
Oct 30 19:42:46.080098 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:46.080074    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-086f2a-9b5f7cf-z2579"]
Oct 30 19:42:46.867894 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:46.865998    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-086f2a-9b5f7cf-z2579" event={"ID":"d925ef89-ce40-4ed1-ad1a-eadd4778245e","Type":"ContainerStarted","Data":"6fd5df0e49899c37be4b643fdbc1e795b2606432a7252a19ba46e5eac073385f"}
Oct 30 19:42:46.867894 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:46.866071    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-086f2a-9b5f7cf-z2579" event={"ID":"d925ef89-ce40-4ed1-ad1a-eadd4778245e","Type":"ContainerStarted","Data":"e5413a4b665606c39fa9a4abaf5ba4f73788c178d5df26014f7946e29d1ca4dc"}
Oct 30 19:42:46.867894 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:46.867200    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-086f2a-9b5f7cf-z2579"
Oct 30 19:42:46.916669 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:46.916615    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-086f2a-9b5f7cf-z2579" podStartSLOduration=2.916598444 podStartE2EDuration="2.916598444s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:42:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:42:46.914833663 +0000 UTC m=+1278.757765612" watchObservedRunningTime="2026-10-30 19:42:46.916598444 +0000 UTC m=+1278.759530377"
Oct 30 19:42:47.535563 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:42:47.535525    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:42:55.260162 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:55.260126    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-34bdc0-kjwss"]
Oct 30 19:42:55.328520 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:55.328490    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-34bdc0-kjwss"]
Oct 30 19:42:56.291077 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:56.291043    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="07e7453d-1591-404e-98ff-6be69a652bd3" path="/var/lib/kubelet/pods/07e7453d-1591-404e-98ff-6be69a652bd3/volumes"
Oct 30 19:42:58.887398 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:42:58.887367    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-086f2a-9b5f7cf-z2579"
Oct 30 19:43:02.043155 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:02.043108    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"]
Oct 30 19:43:02.044038 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:02.044004    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2" podUID="5d44e37a-fe67-40e5-a3d4-c7665f508b76" containerName="istio-proxy" containerID="cri-o://ec97339621ffc11976e39b69fcc5b9876e0d56428d6a2f6ace4f447a1f41a41a" gracePeriod=30
Oct 30 19:43:02.060414 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:02.054583    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"]
Oct 30 19:43:02.060414 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:02.054769    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt" podUID="46317bf7-b022-49a3-b641-66d47bcfb148" containerName="istio-proxy" containerID="cri-o://481e1b2abb4671b891b984deeb682056d2be9d662a15fd8161624d1dae7242ec" gracePeriod=30
Oct 30 19:43:02.073947 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:02.066162    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"]
Oct 30 19:43:02.073947 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:02.066393    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6" podUID="f90bc51f-a0ff-47f9-a334-1a2479c933fb" containerName="istio-proxy" containerID="cri-o://7b9616a4c6ae3b1c8d4fa31c97cbc3ac8b49a47c6e5587696dc44603e63a8907" gracePeriod=30
Oct 30 19:43:02.095691 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:02.095659    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"]
Oct 30 19:43:02.096046 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:02.096006    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm" podUID="ce80464b-ca17-4aed-8228-e19549d3e1de" containerName="istio-proxy" containerID="cri-o://9949530bd3505e441f3bd1cf4907e339892d1b7f86b1e6af4a81678edaf82c5b" gracePeriod=30
Oct 30 19:43:02.104587 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:02.104560    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"]
Oct 30 19:43:02.104818 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:02.104793    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl" podUID="ca6dddc7-2d29-4500-ab3f-7c982fc8064d" containerName="istio-proxy" containerID="cri-o://1ba2acaa95a720fbb49c7b41d62bf366e9d93025619d73e65606f4c39afa6b91" gracePeriod=30
Oct 30 19:43:02.117822 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:02.117111    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"]
Oct 30 19:43:02.117822 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:02.117353    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s" podUID="ddf16588-cce5-46af-8ff3-e927fb5d76b5" containerName="istio-proxy" containerID="cri-o://581fa8b5cf363fd38edafe510db0b783bead95d822bd1496ca2c678f13e29ec9" gracePeriod=30
Oct 30 19:43:02.125325 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:02.125296    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"]
Oct 30 19:43:02.125949 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:02.125699    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc" podUID="f67f5efb-73b5-49fa-b044-36471405ce25" containerName="istio-proxy" containerID="cri-o://fb40a12db00274f0fc33da971134b8ce534acaed2fdd623eef70c8179ab054b3" gracePeriod=30
Oct 30 19:43:02.219478 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:02.219444    5907 patch_prober.go:29] interesting pod/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:43:02.219671 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:02.219497    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2" podUID="5d44e37a-fe67-40e5-a3d4-c7665f508b76" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:43:02.318188 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:02.318069    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:43:02.318188 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:02.318129    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc" podUID="f67f5efb-73b5-49fa-b044-36471405ce25" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:43:02.591134 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:02.590981    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"]
Oct 30 19:43:02.591342 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:02.591311    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc" podUID="34a1c006-cd05-48d7-9cb1-00635b1f4729" containerName="istio-proxy" containerID="cri-o://094132593af5cd182cda41d836026dc341d4ec3a843350b75271762cf774f64f" gracePeriod=30
Oct 30 19:43:02.597245 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:02.597211    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"]
Oct 30 19:43:02.597527 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:02.597494    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv" podUID="1a16775f-a08c-433c-8707-9eed8d73e7f9" containerName="istio-proxy" containerID="cri-o://f715990ad4b0265ef5f435e4c064331b3ceff62f6bbf5da249fdca18ea8e8d65" gracePeriod=30
Oct 30 19:43:02.602122 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:02.602094    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"]
Oct 30 19:43:02.606635 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:02.606611    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"]
Oct 30 19:43:02.606818 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:02.606798    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6" podUID="1ed86722-1b31-4348-ac72-136909f507c3" containerName="istio-proxy" containerID="cri-o://1ffc30a47b8cf11e36e1d111813b6890868c79f41730257fa0381f107777e205" gracePeriod=30
Oct 30 19:43:03.213381 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:03.212758    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5" podUID="819b9937-3bc3-47ac-9631-b6c4fa30a80d" containerName="istio-proxy" containerID="cri-o://e731427a8cd1043751072da437ef799eb543c3b5d60ac0b5602aa9982a7424a9" gracePeriod=30
Oct 30 19:43:05.164357 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:05.164318    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"]
Oct 30 19:43:05.164804 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:05.164604    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6" podUID="76a6fce4-3ba8-434a-b126-8a1a8465b1bf" containerName="istio-proxy" containerID="cri-o://c298eb1f658ad6239263f90a8ee68905b2504b8c78bfb5a34b3fe987eeb766ea" gracePeriod=30
Oct 30 19:43:06.264010 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:06.263976    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0c305f-8677844c-mdcqk"]
Oct 30 19:43:06.270413 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:06.270389    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0c305f-8677844c-mdcqk"
Oct 30 19:43:06.298834 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:06.298801    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0c305f-8677844c-mdcqk"]
Oct 30 19:43:06.306494 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:06.306443    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w0-0c305f-dockercfg-5k4qn\""
Oct 30 19:43:06.306731 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:06.306541    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w0-0c305f\""
Oct 30 19:43:06.350432 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:06.350394    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7e89da-t5hgg"]
Oct 30 19:43:06.432994 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:06.392246    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7e89da-t5hgg"
Oct 30 19:43:06.432994 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:06.431167    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldtk2\" (UniqueName: \"kubernetes.io/projected/998e3875-ef86-4381-9d65-87ca3bba0b1d-kube-api-access-ldtk2\") pod \"payload-pre-processing-e2e-shared-a-w0-0c305f-8677844c-mdcqk\" (UID: \"998e3875-ef86-4381-9d65-87ca3bba0b1d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0c305f-8677844c-mdcqk"
Oct 30 19:43:06.473888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:06.440352    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/998e3875-ef86-4381-9d65-87ca3bba0b1d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-0c305f-8677844c-mdcqk\" (UID: \"998e3875-ef86-4381-9d65-87ca3bba0b1d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0c305f-8677844c-mdcqk"
Oct 30 19:43:06.504890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:06.483332    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:43:06.504890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:06.484278    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:43:06.603005 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:06.590552    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d26qs\" (UniqueName: \"kubernetes.io/projected/713310f5-4d82-4e18-877d-235cfac28327-kube-api-access-d26qs\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-7e89da-t5hgg\" (UID: \"713310f5-4d82-4e18-877d-235cfac28327\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7e89da-t5hgg"
Oct 30 19:43:06.603005 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:06.590761    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/713310f5-4d82-4e18-877d-235cfac28327-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-7e89da-t5hgg\" (UID: \"713310f5-4d82-4e18-877d-235cfac28327\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7e89da-t5hgg"
Oct 30 19:43:06.603005 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:06.590822    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/713310f5-4d82-4e18-877d-235cfac28327-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-7e89da-t5hgg\" (UID: \"713310f5-4d82-4e18-877d-235cfac28327\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7e89da-t5hgg"
Oct 30 19:43:06.603005 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:06.590966    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldtk2\" (UniqueName: \"kubernetes.io/projected/998e3875-ef86-4381-9d65-87ca3bba0b1d-kube-api-access-ldtk2\") pod \"payload-pre-processing-e2e-shared-a-w0-0c305f-8677844c-mdcqk\" (UID: \"998e3875-ef86-4381-9d65-87ca3bba0b1d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0c305f-8677844c-mdcqk"
Oct 30 19:43:06.603005 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:06.591134    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/998e3875-ef86-4381-9d65-87ca3bba0b1d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-0c305f-8677844c-mdcqk\" (UID: \"998e3875-ef86-4381-9d65-87ca3bba0b1d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0c305f-8677844c-mdcqk"
Oct 30 19:43:06.603005 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:06.592176    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/998e3875-ef86-4381-9d65-87ca3bba0b1d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-0c305f-8677844c-mdcqk\" (UID: \"998e3875-ef86-4381-9d65-87ca3bba0b1d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0c305f-8677844c-mdcqk"
Oct 30 19:43:06.731044 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:06.725836    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d26qs\" (UniqueName: \"kubernetes.io/projected/713310f5-4d82-4e18-877d-235cfac28327-kube-api-access-d26qs\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-7e89da-t5hgg\" (UID: \"713310f5-4d82-4e18-877d-235cfac28327\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7e89da-t5hgg"
Oct 30 19:43:06.739888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:06.731558    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/713310f5-4d82-4e18-877d-235cfac28327-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-7e89da-t5hgg\" (UID: \"713310f5-4d82-4e18-877d-235cfac28327\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7e89da-t5hgg"
Oct 30 19:43:06.743302 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:06.743268    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/713310f5-4d82-4e18-877d-235cfac28327-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-7e89da-t5hgg\" (UID: \"713310f5-4d82-4e18-877d-235cfac28327\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7e89da-t5hgg"
Oct 30 19:43:06.744754 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:06.744720    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/713310f5-4d82-4e18-877d-235cfac28327-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-7e89da-t5hgg\" (UID: \"713310f5-4d82-4e18-877d-235cfac28327\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7e89da-t5hgg"
Oct 30 19:43:06.757885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:06.753955    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7e89da-t5hgg"]
Oct 30 19:43:06.769274 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:06.769205    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/713310f5-4d82-4e18-877d-235cfac28327-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-7e89da-t5hgg\" (UID: \"713310f5-4d82-4e18-877d-235cfac28327\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7e89da-t5hgg"
Oct 30 19:43:06.772610 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:06.772545    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d26qs\" (UniqueName: \"kubernetes.io/projected/713310f5-4d82-4e18-877d-235cfac28327-kube-api-access-d26qs\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-7e89da-t5hgg\" (UID: \"713310f5-4d82-4e18-877d-235cfac28327\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7e89da-t5hgg"
Oct 30 19:43:06.897960 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:06.877080    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-0c305f-5f5c4698b6-kzfnv"]
Oct 30 19:43:06.897960 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:06.888986    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7e89da-t5hgg"
Oct 30 19:43:06.897960 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:06.889072    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldtk2\" (UniqueName: \"kubernetes.io/projected/998e3875-ef86-4381-9d65-87ca3bba0b1d-kube-api-access-ldtk2\") pod \"payload-pre-processing-e2e-shared-a-w0-0c305f-8677844c-mdcqk\" (UID: \"998e3875-ef86-4381-9d65-87ca3bba0b1d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0c305f-8677844c-mdcqk"
Oct 30 19:43:06.897960 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:06.889620    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0c305f-8677844c-mdcqk"
Oct 30 19:43:06.914327 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:06.914307    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-0c305f-5f5c4698b6-kzfnv"]
Oct 30 19:43:06.915304 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:06.915240    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0c305f-5f5c4698b6-kzfnv"
Oct 30 19:43:07.066756 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.066719    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8pdz\" (UniqueName: \"kubernetes.io/projected/400f0778-d2b1-4ca0-93ec-59672a71e661-kube-api-access-z8pdz\") pod \"payload-processing-e2e-shared-a-w0-0c305f-5f5c4698b6-kzfnv\" (UID: \"400f0778-d2b1-4ca0-93ec-59672a71e661\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0c305f-5f5c4698b6-kzfnv"
Oct 30 19:43:07.066925 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.066822    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/400f0778-d2b1-4ca0-93ec-59672a71e661-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-0c305f-5f5c4698b6-kzfnv\" (UID: \"400f0778-d2b1-4ca0-93ec-59672a71e661\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0c305f-5f5c4698b6-kzfnv"
Oct 30 19:43:07.168717 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.168687    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8pdz\" (UniqueName: \"kubernetes.io/projected/400f0778-d2b1-4ca0-93ec-59672a71e661-kube-api-access-z8pdz\") pod \"payload-processing-e2e-shared-a-w0-0c305f-5f5c4698b6-kzfnv\" (UID: \"400f0778-d2b1-4ca0-93ec-59672a71e661\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0c305f-5f5c4698b6-kzfnv"
Oct 30 19:43:07.168829 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.168767    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/400f0778-d2b1-4ca0-93ec-59672a71e661-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-0c305f-5f5c4698b6-kzfnv\" (UID: \"400f0778-d2b1-4ca0-93ec-59672a71e661\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0c305f-5f5c4698b6-kzfnv"
Oct 30 19:43:07.169567 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.169546    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/400f0778-d2b1-4ca0-93ec-59672a71e661-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-0c305f-5f5c4698b6-kzfnv\" (UID: \"400f0778-d2b1-4ca0-93ec-59672a71e661\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0c305f-5f5c4698b6-kzfnv"
Oct 30 19:43:07.206384 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.206356    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8pdz\" (UniqueName: \"kubernetes.io/projected/400f0778-d2b1-4ca0-93ec-59672a71e661-kube-api-access-z8pdz\") pod \"payload-processing-e2e-shared-a-w0-0c305f-5f5c4698b6-kzfnv\" (UID: \"400f0778-d2b1-4ca0-93ec-59672a71e661\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0c305f-5f5c4698b6-kzfnv"
Oct 30 19:43:07.416398 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.410586    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0c305f-5f5c4698b6-kzfnv"
Oct 30 19:43:07.441129 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:43:07.424365    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod998e3875_ef86_4381_9d65_87ca3bba0b1d.slice/crio-dd0f4526b83f481459f47a4fba5ef4d8fa3c393d7ed26f3edf9828c379055888 WatchSource:0}: Error finding container dd0f4526b83f481459f47a4fba5ef4d8fa3c393d7ed26f3edf9828c379055888: Status 404 returned error can't find the container with id dd0f4526b83f481459f47a4fba5ef4d8fa3c393d7ed26f3edf9828c379055888
Oct 30 19:43:07.441129 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.434206    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7e89da-t5hgg"]
Oct 30 19:43:07.441129 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.438755    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0c305f-8677844c-mdcqk"]
Oct 30 19:43:07.536234 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.536090    5907 generic.go:356] "Generic (PLEG): container finished" podID="f90bc51f-a0ff-47f9-a334-1a2479c933fb" containerID="7b9616a4c6ae3b1c8d4fa31c97cbc3ac8b49a47c6e5587696dc44603e63a8907" exitCode=0
Oct 30 19:43:07.536234 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.536181    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6" event={"ID":"f90bc51f-a0ff-47f9-a334-1a2479c933fb","Type":"ContainerDied","Data":"7b9616a4c6ae3b1c8d4fa31c97cbc3ac8b49a47c6e5587696dc44603e63a8907"}
Oct 30 19:43:07.542286 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.542015    5907 generic.go:356] "Generic (PLEG): container finished" podID="5d44e37a-fe67-40e5-a3d4-c7665f508b76" containerID="ec97339621ffc11976e39b69fcc5b9876e0d56428d6a2f6ace4f447a1f41a41a" exitCode=0
Oct 30 19:43:07.542286 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.542112    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2" event={"ID":"5d44e37a-fe67-40e5-a3d4-c7665f508b76","Type":"ContainerDied","Data":"ec97339621ffc11976e39b69fcc5b9876e0d56428d6a2f6ace4f447a1f41a41a"}
Oct 30 19:43:07.545187 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.545083    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7e89da-t5hgg" event={"ID":"713310f5-4d82-4e18-877d-235cfac28327","Type":"ContainerStarted","Data":"e3b9f627a3504c7d1ed1ee4b582cc45431bb280984ea0366a26b5ccc1f2f360f"}
Oct 30 19:43:07.548592 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.548490    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0c305f-8677844c-mdcqk" event={"ID":"998e3875-ef86-4381-9d65-87ca3bba0b1d","Type":"ContainerStarted","Data":"dd0f4526b83f481459f47a4fba5ef4d8fa3c393d7ed26f3edf9828c379055888"}
Oct 30 19:43:07.556830 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.556002    5907 generic.go:356] "Generic (PLEG): container finished" podID="46317bf7-b022-49a3-b641-66d47bcfb148" containerID="481e1b2abb4671b891b984deeb682056d2be9d662a15fd8161624d1dae7242ec" exitCode=0
Oct 30 19:43:07.557762 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.557657    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt" event={"ID":"46317bf7-b022-49a3-b641-66d47bcfb148","Type":"ContainerDied","Data":"481e1b2abb4671b891b984deeb682056d2be9d662a15fd8161624d1dae7242ec"}
Oct 30 19:43:07.563485 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.563305    5907 generic.go:356] "Generic (PLEG): container finished" podID="f67f5efb-73b5-49fa-b044-36471405ce25" containerID="fb40a12db00274f0fc33da971134b8ce534acaed2fdd623eef70c8179ab054b3" exitCode=0
Oct 30 19:43:07.563485 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.563398    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc" event={"ID":"f67f5efb-73b5-49fa-b044-36471405ce25","Type":"ContainerDied","Data":"fb40a12db00274f0fc33da971134b8ce534acaed2fdd623eef70c8179ab054b3"}
Oct 30 19:43:07.565837 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.565627    5907 generic.go:356] "Generic (PLEG): container finished" podID="ddf16588-cce5-46af-8ff3-e927fb5d76b5" containerID="581fa8b5cf363fd38edafe510db0b783bead95d822bd1496ca2c678f13e29ec9" exitCode=0
Oct 30 19:43:07.565837 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.565765    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s" event={"ID":"ddf16588-cce5-46af-8ff3-e927fb5d76b5","Type":"ContainerDied","Data":"581fa8b5cf363fd38edafe510db0b783bead95d822bd1496ca2c678f13e29ec9"}
Oct 30 19:43:07.569224 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.569202    5907 generic.go:356] "Generic (PLEG): container finished" podID="ce80464b-ca17-4aed-8228-e19549d3e1de" containerID="9949530bd3505e441f3bd1cf4907e339892d1b7f86b1e6af4a81678edaf82c5b" exitCode=0
Oct 30 19:43:07.569434 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.569418    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm" event={"ID":"ce80464b-ca17-4aed-8228-e19549d3e1de","Type":"ContainerDied","Data":"9949530bd3505e441f3bd1cf4907e339892d1b7f86b1e6af4a81678edaf82c5b"}
Oct 30 19:43:07.576357 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.576315    5907 generic.go:356] "Generic (PLEG): container finished" podID="ca6dddc7-2d29-4500-ab3f-7c982fc8064d" containerID="1ba2acaa95a720fbb49c7b41d62bf366e9d93025619d73e65606f4c39afa6b91" exitCode=0
Oct 30 19:43:07.576839 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.576735    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl" event={"ID":"ca6dddc7-2d29-4500-ab3f-7c982fc8064d","Type":"ContainerDied","Data":"1ba2acaa95a720fbb49c7b41d62bf366e9d93025619d73e65606f4c39afa6b91"}
Oct 30 19:43:07.577029 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.577017    5907 scope.go:122] "RemoveContainer" containerID="e0b5013aeda8e3120b40f658b4ca5bbce6b6b3cd404cf13c3c06bdd26c6d88e7"
Oct 30 19:43:07.911952 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.911675    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"
Oct 30 19:43:07.958934 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.958820    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"
Oct 30 19:43:07.964250 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.964220    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:43:07.969595 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.969565    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"
Oct 30 19:43:07.970263 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.970231    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/46317bf7-b022-49a3-b641-66d47bcfb148-istio-token\" (UniqueName: \"kubernetes.io/projected/46317bf7-b022-49a3-b641-66d47bcfb148-istio-token\") pod \"46317bf7-b022-49a3-b641-66d47bcfb148\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") "
Oct 30 19:43:07.970404 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.970287    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46317bf7-b022-49a3-b641-66d47bcfb148-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46317bf7-b022-49a3-b641-66d47bcfb148-credential-socket\") pod \"46317bf7-b022-49a3-b641-66d47bcfb148\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") "
Oct 30 19:43:07.970489 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.970466    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46317bf7-b022-49a3-b641-66d47bcfb148-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46317bf7-b022-49a3-b641-66d47bcfb148-workload-certs\") pod \"46317bf7-b022-49a3-b641-66d47bcfb148\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") "
Oct 30 19:43:07.970567 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.970551    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/46317bf7-b022-49a3-b641-66d47bcfb148-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46317bf7-b022-49a3-b641-66d47bcfb148-istio-podinfo\") pod \"46317bf7-b022-49a3-b641-66d47bcfb148\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") "
Oct 30 19:43:07.970688 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.970656    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46317bf7-b022-49a3-b641-66d47bcfb148-credential-socket" pod "46317bf7-b022-49a3-b641-66d47bcfb148" (UID: "46317bf7-b022-49a3-b641-66d47bcfb148"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:07.970807 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.970687    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46317bf7-b022-49a3-b641-66d47bcfb148-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46317bf7-b022-49a3-b641-66d47bcfb148-workload-socket\") pod \"46317bf7-b022-49a3-b641-66d47bcfb148\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") "
Oct 30 19:43:07.970807 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.970738    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46317bf7-b022-49a3-b641-66d47bcfb148-workload-certs" pod "46317bf7-b022-49a3-b641-66d47bcfb148" (UID: "46317bf7-b022-49a3-b641-66d47bcfb148"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:07.970807 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.970798    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46317bf7-b022-49a3-b641-66d47bcfb148-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46317bf7-b022-49a3-b641-66d47bcfb148-istio-envoy\") pod \"46317bf7-b022-49a3-b641-66d47bcfb148\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") "
Oct 30 19:43:07.971042 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.970835    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/46317bf7-b022-49a3-b641-66d47bcfb148-kube-api-access-j7sfj\" (UniqueName: \"kubernetes.io/projected/46317bf7-b022-49a3-b641-66d47bcfb148-kube-api-access-j7sfj\") pod \"46317bf7-b022-49a3-b641-66d47bcfb148\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") "
Oct 30 19:43:07.971042 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.970903    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46317bf7-b022-49a3-b641-66d47bcfb148-workload-socket" pod "46317bf7-b022-49a3-b641-66d47bcfb148" (UID: "46317bf7-b022-49a3-b641-66d47bcfb148"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:07.971042 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.970957    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/46317bf7-b022-49a3-b641-66d47bcfb148-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46317bf7-b022-49a3-b641-66d47bcfb148-istiod-ca-cert\") pod \"46317bf7-b022-49a3-b641-66d47bcfb148\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") "
Oct 30 19:43:07.971201 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.971117    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46317bf7-b022-49a3-b641-66d47bcfb148-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46317bf7-b022-49a3-b641-66d47bcfb148-istio-data\") pod \"46317bf7-b022-49a3-b641-66d47bcfb148\" (UID: \"46317bf7-b022-49a3-b641-66d47bcfb148\") "
Oct 30 19:43:07.971467 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.971439    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46317bf7-b022-49a3-b641-66d47bcfb148-istio-data" pod "46317bf7-b022-49a3-b641-66d47bcfb148" (UID: "46317bf7-b022-49a3-b641-66d47bcfb148"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:07.972024 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.971998    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46317bf7-b022-49a3-b641-66d47bcfb148-istiod-ca-cert" pod "46317bf7-b022-49a3-b641-66d47bcfb148" (UID: "46317bf7-b022-49a3-b641-66d47bcfb148"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:07.973124 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.973100    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46317bf7-b022-49a3-b641-66d47bcfb148-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:07.973251 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.973128    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46317bf7-b022-49a3-b641-66d47bcfb148-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:07.973251 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.973141    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46317bf7-b022-49a3-b641-66d47bcfb148-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:07.973251 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.973158    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46317bf7-b022-49a3-b641-66d47bcfb148-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:07.973251 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.973173    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46317bf7-b022-49a3-b641-66d47bcfb148-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:07.976069 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:07.976049    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"
Oct 30 19:43:08.007684 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.007660    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"
Oct 30 19:43:08.010479 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.010451    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46317bf7-b022-49a3-b641-66d47bcfb148-kube-api-access-j7sfj" pod "46317bf7-b022-49a3-b641-66d47bcfb148" (UID: "46317bf7-b022-49a3-b641-66d47bcfb148"). InnerVolumeSpecName "kube-api-access-j7sfj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:08.010613 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.010496    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46317bf7-b022-49a3-b641-66d47bcfb148-istio-token" pod "46317bf7-b022-49a3-b641-66d47bcfb148" (UID: "46317bf7-b022-49a3-b641-66d47bcfb148"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:08.010613 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.010603    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46317bf7-b022-49a3-b641-66d47bcfb148-istio-envoy" pod "46317bf7-b022-49a3-b641-66d47bcfb148" (UID: "46317bf7-b022-49a3-b641-66d47bcfb148"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.014335 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.014305    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/46317bf7-b022-49a3-b641-66d47bcfb148-istio-podinfo" pod "46317bf7-b022-49a3-b641-66d47bcfb148" (UID: "46317bf7-b022-49a3-b641-66d47bcfb148"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:43:08.015144 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.014769    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"
Oct 30 19:43:08.015144 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.014775    5907 patch_prober.go:29] interesting pod/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:43:08.015144 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.014900    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5" podUID="819b9937-3bc3-47ac-9631-b6c4fa30a80d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:43:08.074050 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.074019    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce80464b-ca17-4aed-8228-e19549d3e1de-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce80464b-ca17-4aed-8228-e19549d3e1de-istio-envoy\") pod \"ce80464b-ca17-4aed-8228-e19549d3e1de\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") "
Oct 30 19:43:08.074050 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.074054    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f90bc51f-a0ff-47f9-a334-1a2479c933fb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f90bc51f-a0ff-47f9-a334-1a2479c933fb-credential-socket\") pod \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") "
Oct 30 19:43:08.074292 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.074088    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f90bc51f-a0ff-47f9-a334-1a2479c933fb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f90bc51f-a0ff-47f9-a334-1a2479c933fb-istiod-ca-cert\") pod \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") "
Oct 30 19:43:08.074292 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.074145    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-istio-envoy\") pod \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") "
Oct 30 19:43:08.074292 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.074178    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f67f5efb-73b5-49fa-b044-36471405ce25-istio-token\" (UniqueName: \"kubernetes.io/projected/f67f5efb-73b5-49fa-b044-36471405ce25-istio-token\") pod \"f67f5efb-73b5-49fa-b044-36471405ce25\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") "
Oct 30 19:43:08.074292 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.074219    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ddf16588-cce5-46af-8ff3-e927fb5d76b5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddf16588-cce5-46af-8ff3-e927fb5d76b5-workload-certs\") pod \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") "
Oct 30 19:43:08.074292 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.074255    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d44e37a-fe67-40e5-a3d4-c7665f508b76-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d44e37a-fe67-40e5-a3d4-c7665f508b76-credential-socket\") pod \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") "
Oct 30 19:43:08.074292 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.074283    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ddf16588-cce5-46af-8ff3-e927fb5d76b5-kube-api-access-g2lb5\" (UniqueName: \"kubernetes.io/projected/ddf16588-cce5-46af-8ff3-e927fb5d76b5-kube-api-access-g2lb5\") pod \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") "
Oct 30 19:43:08.074562 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.074323    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f90bc51f-a0ff-47f9-a334-1a2479c933fb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f90bc51f-a0ff-47f9-a334-1a2479c933fb-workload-certs\") pod \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") "
Oct 30 19:43:08.074562 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.074356    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-workload-certs\") pod \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") "
Oct 30 19:43:08.074562 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.074400    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5d44e37a-fe67-40e5-a3d4-c7665f508b76-kube-api-access-c54js\" (UniqueName: \"kubernetes.io/projected/5d44e37a-fe67-40e5-a3d4-c7665f508b76-kube-api-access-c54js\") pod \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") "
Oct 30 19:43:08.074562 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.074429    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5d44e37a-fe67-40e5-a3d4-c7665f508b76-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d44e37a-fe67-40e5-a3d4-c7665f508b76-istio-podinfo\") pod \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") "
Oct 30 19:43:08.074562 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.074463    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ddf16588-cce5-46af-8ff3-e927fb5d76b5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddf16588-cce5-46af-8ff3-e927fb5d76b5-istio-data\") pod \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") "
Oct 30 19:43:08.074562 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.074511    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ddf16588-cce5-46af-8ff3-e927fb5d76b5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddf16588-cce5-46af-8ff3-e927fb5d76b5-istiod-ca-cert\") pod \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") "
Oct 30 19:43:08.074562 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.074550    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ce80464b-ca17-4aed-8228-e19549d3e1de-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce80464b-ca17-4aed-8228-e19549d3e1de-istiod-ca-cert\") pod \"ce80464b-ca17-4aed-8228-e19549d3e1de\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") "
Oct 30 19:43:08.074972 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.074585    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-workload-socket\") pod \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") "
Oct 30 19:43:08.074972 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.074655    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce80464b-ca17-4aed-8228-e19549d3e1de-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce80464b-ca17-4aed-8228-e19549d3e1de-istio-data\") pod \"ce80464b-ca17-4aed-8228-e19549d3e1de\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") "
Oct 30 19:43:08.074972 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.074702    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5d44e37a-fe67-40e5-a3d4-c7665f508b76-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d44e37a-fe67-40e5-a3d4-c7665f508b76-istiod-ca-cert\") pod \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") "
Oct 30 19:43:08.074972 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.074734    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-istio-data\") pod \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") "
Oct 30 19:43:08.074972 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.074780    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f90bc51f-a0ff-47f9-a334-1a2479c933fb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f90bc51f-a0ff-47f9-a334-1a2479c933fb-istio-data\") pod \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") "
Oct 30 19:43:08.074972 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.074817    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f67f5efb-73b5-49fa-b044-36471405ce25-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f67f5efb-73b5-49fa-b044-36471405ce25-workload-certs\") pod \"f67f5efb-73b5-49fa-b044-36471405ce25\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") "
Oct 30 19:43:08.074972 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.074892    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ddf16588-cce5-46af-8ff3-e927fb5d76b5-istio-token\" (UniqueName: \"kubernetes.io/projected/ddf16588-cce5-46af-8ff3-e927fb5d76b5-istio-token\") pod \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") "
Oct 30 19:43:08.074972 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.074928    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d44e37a-fe67-40e5-a3d4-c7665f508b76-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d44e37a-fe67-40e5-a3d4-c7665f508b76-istio-envoy\") pod \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") "
Oct 30 19:43:08.074972 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.074957    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f67f5efb-73b5-49fa-b044-36471405ce25-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f67f5efb-73b5-49fa-b044-36471405ce25-istio-podinfo\") pod \"f67f5efb-73b5-49fa-b044-36471405ce25\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") "
Oct 30 19:43:08.075422 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.074994    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ddf16588-cce5-46af-8ff3-e927fb5d76b5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddf16588-cce5-46af-8ff3-e927fb5d76b5-istio-podinfo\") pod \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") "
Oct 30 19:43:08.075422 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.075029    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ce80464b-ca17-4aed-8228-e19549d3e1de-istio-token\" (UniqueName: \"kubernetes.io/projected/ce80464b-ca17-4aed-8228-e19549d3e1de-istio-token\") pod \"ce80464b-ca17-4aed-8228-e19549d3e1de\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") "
Oct 30 19:43:08.075422 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.075077    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ce80464b-ca17-4aed-8228-e19549d3e1de-kube-api-access-hfhh4\" (UniqueName: \"kubernetes.io/projected/ce80464b-ca17-4aed-8228-e19549d3e1de-kube-api-access-hfhh4\") pod \"ce80464b-ca17-4aed-8228-e19549d3e1de\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") "
Oct 30 19:43:08.075422 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.075113    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f67f5efb-73b5-49fa-b044-36471405ce25-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f67f5efb-73b5-49fa-b044-36471405ce25-credential-socket\") pod \"f67f5efb-73b5-49fa-b044-36471405ce25\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") "
Oct 30 19:43:08.075422 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.075180    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f90bc51f-a0ff-47f9-a334-1a2479c933fb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f90bc51f-a0ff-47f9-a334-1a2479c933fb-workload-socket\") pod \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") "
Oct 30 19:43:08.075422 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.075207    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f67f5efb-73b5-49fa-b044-36471405ce25-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f67f5efb-73b5-49fa-b044-36471405ce25-istio-data\") pod \"f67f5efb-73b5-49fa-b044-36471405ce25\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") "
Oct 30 19:43:08.075422 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.075357    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f67f5efb-73b5-49fa-b044-36471405ce25-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f67f5efb-73b5-49fa-b044-36471405ce25-istiod-ca-cert\") pod \"f67f5efb-73b5-49fa-b044-36471405ce25\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") "
Oct 30 19:43:08.075945 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.075439    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d44e37a-fe67-40e5-a3d4-c7665f508b76-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d44e37a-fe67-40e5-a3d4-c7665f508b76-workload-socket\") pod \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") "
Oct 30 19:43:08.075945 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.075583    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-istiod-ca-cert\") pod \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") "
Oct 30 19:43:08.075945 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.075622    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f67f5efb-73b5-49fa-b044-36471405ce25-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f67f5efb-73b5-49fa-b044-36471405ce25-workload-socket\") pod \"f67f5efb-73b5-49fa-b044-36471405ce25\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") "
Oct 30 19:43:08.075945 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.075833    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce80464b-ca17-4aed-8228-e19549d3e1de-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce80464b-ca17-4aed-8228-e19549d3e1de-workload-certs\") pod \"ce80464b-ca17-4aed-8228-e19549d3e1de\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") "
Oct 30 19:43:08.075945 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.075921    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f90bc51f-a0ff-47f9-a334-1a2479c933fb-kube-api-access-zfgl5\" (UniqueName: \"kubernetes.io/projected/f90bc51f-a0ff-47f9-a334-1a2479c933fb-kube-api-access-zfgl5\") pod \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") "
Oct 30 19:43:08.076205 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.075966    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d44e37a-fe67-40e5-a3d4-c7665f508b76-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d44e37a-fe67-40e5-a3d4-c7665f508b76-workload-certs\") pod \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") "
Oct 30 19:43:08.076205 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.076043    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce80464b-ca17-4aed-8228-e19549d3e1de-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce80464b-ca17-4aed-8228-e19549d3e1de-credential-socket\") pod \"ce80464b-ca17-4aed-8228-e19549d3e1de\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") "
Oct 30 19:43:08.076205 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.076066    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-credential-socket\") pod \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") "
Oct 30 19:43:08.076205 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.076096    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-istio-podinfo\") pod \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") "
Oct 30 19:43:08.076205 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.076124    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-kube-api-access-xm9l5\" (UniqueName: \"kubernetes.io/projected/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-kube-api-access-xm9l5\") pod \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") "
Oct 30 19:43:08.076205 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.076162    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ddf16588-cce5-46af-8ff3-e927fb5d76b5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddf16588-cce5-46af-8ff3-e927fb5d76b5-workload-socket\") pod \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") "
Oct 30 19:43:08.076702 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.076221    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce80464b-ca17-4aed-8228-e19549d3e1de-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce80464b-ca17-4aed-8228-e19549d3e1de-workload-socket\") pod \"ce80464b-ca17-4aed-8228-e19549d3e1de\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") "
Oct 30 19:43:08.076702 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.076264    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f90bc51f-a0ff-47f9-a334-1a2479c933fb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f90bc51f-a0ff-47f9-a334-1a2479c933fb-istio-podinfo\") pod \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") "
Oct 30 19:43:08.076702 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.076322    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d44e37a-fe67-40e5-a3d4-c7665f508b76-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d44e37a-fe67-40e5-a3d4-c7665f508b76-istio-data\") pod \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") "
Oct 30 19:43:08.076702 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.076364    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f67f5efb-73b5-49fa-b044-36471405ce25-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f67f5efb-73b5-49fa-b044-36471405ce25-istio-envoy\") pod \"f67f5efb-73b5-49fa-b044-36471405ce25\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") "
Oct 30 19:43:08.076702 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.076424    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f67f5efb-73b5-49fa-b044-36471405ce25-kube-api-access-fjdps\" (UniqueName: \"kubernetes.io/projected/f67f5efb-73b5-49fa-b044-36471405ce25-kube-api-access-fjdps\") pod \"f67f5efb-73b5-49fa-b044-36471405ce25\" (UID: \"f67f5efb-73b5-49fa-b044-36471405ce25\") "
Oct 30 19:43:08.076702 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.076485    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f90bc51f-a0ff-47f9-a334-1a2479c933fb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f90bc51f-a0ff-47f9-a334-1a2479c933fb-istio-envoy\") pod \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") "
Oct 30 19:43:08.076702 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.076517    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ddf16588-cce5-46af-8ff3-e927fb5d76b5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddf16588-cce5-46af-8ff3-e927fb5d76b5-credential-socket\") pod \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") "
Oct 30 19:43:08.076702 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.076601    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5d44e37a-fe67-40e5-a3d4-c7665f508b76-istio-token\" (UniqueName: \"kubernetes.io/projected/5d44e37a-fe67-40e5-a3d4-c7665f508b76-istio-token\") pod \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\" (UID: \"5d44e37a-fe67-40e5-a3d4-c7665f508b76\") "
Oct 30 19:43:08.077971 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.076784    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ce80464b-ca17-4aed-8228-e19549d3e1de-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce80464b-ca17-4aed-8228-e19549d3e1de-istio-podinfo\") pod \"ce80464b-ca17-4aed-8228-e19549d3e1de\" (UID: \"ce80464b-ca17-4aed-8228-e19549d3e1de\") "
Oct 30 19:43:08.077971 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.076814    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f90bc51f-a0ff-47f9-a334-1a2479c933fb-istio-token\" (UniqueName: \"kubernetes.io/projected/f90bc51f-a0ff-47f9-a334-1a2479c933fb-istio-token\") pod \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\" (UID: \"f90bc51f-a0ff-47f9-a334-1a2479c933fb\") "
Oct 30 19:43:08.077971 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.076846    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ddf16588-cce5-46af-8ff3-e927fb5d76b5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddf16588-cce5-46af-8ff3-e927fb5d76b5-istio-envoy\") pod \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\" (UID: \"ddf16588-cce5-46af-8ff3-e927fb5d76b5\") "
Oct 30 19:43:08.077971 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.076920    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-istio-token\" (UniqueName: \"kubernetes.io/projected/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-istio-token\") pod \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\" (UID: \"ca6dddc7-2d29-4500-ab3f-7c982fc8064d\") "
Oct 30 19:43:08.078302 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.078278    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46317bf7-b022-49a3-b641-66d47bcfb148-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.078387 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.078310    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46317bf7-b022-49a3-b641-66d47bcfb148-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.078387 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.078327    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7sfj\" (UniqueName: \"kubernetes.io/projected/46317bf7-b022-49a3-b641-66d47bcfb148-kube-api-access-j7sfj\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.078387 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.078342    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46317bf7-b022-49a3-b641-66d47bcfb148-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.080682 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.074354    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f90bc51f-a0ff-47f9-a334-1a2479c933fb-credential-socket" pod "f90bc51f-a0ff-47f9-a334-1a2479c933fb" (UID: "f90bc51f-a0ff-47f9-a334-1a2479c933fb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.080804 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.074613    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddf16588-cce5-46af-8ff3-e927fb5d76b5-workload-certs" pod "ddf16588-cce5-46af-8ff3-e927fb5d76b5" (UID: "ddf16588-cce5-46af-8ff3-e927fb5d76b5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.080804 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.074640    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f90bc51f-a0ff-47f9-a334-1a2479c933fb-workload-certs" pod "f90bc51f-a0ff-47f9-a334-1a2479c933fb" (UID: "f90bc51f-a0ff-47f9-a334-1a2479c933fb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.080804 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.074834    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d44e37a-fe67-40e5-a3d4-c7665f508b76-credential-socket" pod "5d44e37a-fe67-40e5-a3d4-c7665f508b76" (UID: "5d44e37a-fe67-40e5-a3d4-c7665f508b76"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.080804 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.075199    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce80464b-ca17-4aed-8228-e19549d3e1de-istiod-ca-cert" pod "ce80464b-ca17-4aed-8228-e19549d3e1de" (UID: "ce80464b-ca17-4aed-8228-e19549d3e1de"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:08.080804 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.075681    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f90bc51f-a0ff-47f9-a334-1a2479c933fb-istiod-ca-cert" pod "f90bc51f-a0ff-47f9-a334-1a2479c933fb" (UID: "f90bc51f-a0ff-47f9-a334-1a2479c933fb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:08.080804 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.080464    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d44e37a-fe67-40e5-a3d4-c7665f508b76-kube-api-access-c54js" pod "5d44e37a-fe67-40e5-a3d4-c7665f508b76" (UID: "5d44e37a-fe67-40e5-a3d4-c7665f508b76"). InnerVolumeSpecName "kube-api-access-c54js". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:08.080804 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.080653    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-workload-certs" pod "ca6dddc7-2d29-4500-ab3f-7c982fc8064d" (UID: "ca6dddc7-2d29-4500-ab3f-7c982fc8064d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.081610 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.081586    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5d44e37a-fe67-40e5-a3d4-c7665f508b76-istio-podinfo" pod "5d44e37a-fe67-40e5-a3d4-c7665f508b76" (UID: "5d44e37a-fe67-40e5-a3d4-c7665f508b76"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:43:08.081941 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.081921    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddf16588-cce5-46af-8ff3-e927fb5d76b5-istio-data" pod "ddf16588-cce5-46af-8ff3-e927fb5d76b5" (UID: "ddf16588-cce5-46af-8ff3-e927fb5d76b5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.082377 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.082349    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddf16588-cce5-46af-8ff3-e927fb5d76b5-istiod-ca-cert" pod "ddf16588-cce5-46af-8ff3-e927fb5d76b5" (UID: "ddf16588-cce5-46af-8ff3-e927fb5d76b5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:08.082974 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.082955    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-istio-data" pod "ca6dddc7-2d29-4500-ab3f-7c982fc8064d" (UID: "ca6dddc7-2d29-4500-ab3f-7c982fc8064d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.083470 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.083445    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-workload-socket" pod "ca6dddc7-2d29-4500-ab3f-7c982fc8064d" (UID: "ca6dddc7-2d29-4500-ab3f-7c982fc8064d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.083664 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.083643    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce80464b-ca17-4aed-8228-e19549d3e1de-istio-data" pod "ce80464b-ca17-4aed-8228-e19549d3e1de" (UID: "ce80464b-ca17-4aed-8228-e19549d3e1de"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.083961 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.083941    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d44e37a-fe67-40e5-a3d4-c7665f508b76-istiod-ca-cert" pod "5d44e37a-fe67-40e5-a3d4-c7665f508b76" (UID: "5d44e37a-fe67-40e5-a3d4-c7665f508b76"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:08.084183 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.084163    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-credential-socket" pod "ca6dddc7-2d29-4500-ab3f-7c982fc8064d" (UID: "ca6dddc7-2d29-4500-ab3f-7c982fc8064d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.084942 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.084916    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddf16588-cce5-46af-8ff3-e927fb5d76b5-kube-api-access-g2lb5" pod "ddf16588-cce5-46af-8ff3-e927fb5d76b5" (UID: "ddf16588-cce5-46af-8ff3-e927fb5d76b5"). InnerVolumeSpecName "kube-api-access-g2lb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:08.086344 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.085991    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d44e37a-fe67-40e5-a3d4-c7665f508b76-workload-socket" pod "5d44e37a-fe67-40e5-a3d4-c7665f508b76" (UID: "5d44e37a-fe67-40e5-a3d4-c7665f508b76"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.086344 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.086070    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-istio-token" pod "ca6dddc7-2d29-4500-ab3f-7c982fc8064d" (UID: "ca6dddc7-2d29-4500-ab3f-7c982fc8064d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:08.086344 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.086111    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce80464b-ca17-4aed-8228-e19549d3e1de-istio-envoy" pod "ce80464b-ca17-4aed-8228-e19549d3e1de" (UID: "ce80464b-ca17-4aed-8228-e19549d3e1de"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.086655 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.086354    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f67f5efb-73b5-49fa-b044-36471405ce25-workload-certs" pod "f67f5efb-73b5-49fa-b044-36471405ce25" (UID: "f67f5efb-73b5-49fa-b044-36471405ce25"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.086655 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.086528    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f90bc51f-a0ff-47f9-a334-1a2479c933fb-istio-data" pod "f90bc51f-a0ff-47f9-a334-1a2479c933fb" (UID: "f90bc51f-a0ff-47f9-a334-1a2479c933fb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.086762 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.086678    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f90bc51f-a0ff-47f9-a334-1a2479c933fb-workload-socket" pod "f90bc51f-a0ff-47f9-a334-1a2479c933fb" (UID: "f90bc51f-a0ff-47f9-a334-1a2479c933fb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.086762 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.086741    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-istio-envoy" pod "ca6dddc7-2d29-4500-ab3f-7c982fc8064d" (UID: "ca6dddc7-2d29-4500-ab3f-7c982fc8064d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.087581 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.087325    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f67f5efb-73b5-49fa-b044-36471405ce25-istio-podinfo" pod "f67f5efb-73b5-49fa-b044-36471405ce25" (UID: "f67f5efb-73b5-49fa-b044-36471405ce25"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:43:08.092357 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.088205    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f67f5efb-73b5-49fa-b044-36471405ce25-credential-socket" pod "f67f5efb-73b5-49fa-b044-36471405ce25" (UID: "f67f5efb-73b5-49fa-b044-36471405ce25"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.092357 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.088417    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce80464b-ca17-4aed-8228-e19549d3e1de-workload-certs" pod "ce80464b-ca17-4aed-8228-e19549d3e1de" (UID: "ce80464b-ca17-4aed-8228-e19549d3e1de"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.092357 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.088620    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f67f5efb-73b5-49fa-b044-36471405ce25-istio-token" pod "f67f5efb-73b5-49fa-b044-36471405ce25" (UID: "f67f5efb-73b5-49fa-b044-36471405ce25"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:08.092357 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.088804    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d44e37a-fe67-40e5-a3d4-c7665f508b76-workload-certs" pod "5d44e37a-fe67-40e5-a3d4-c7665f508b76" (UID: "5d44e37a-fe67-40e5-a3d4-c7665f508b76"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.092357 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.089353    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce80464b-ca17-4aed-8228-e19549d3e1de-credential-socket" pod "ce80464b-ca17-4aed-8228-e19549d3e1de" (UID: "ce80464b-ca17-4aed-8228-e19549d3e1de"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.092357 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.092171    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce80464b-ca17-4aed-8228-e19549d3e1de-kube-api-access-hfhh4" pod "ce80464b-ca17-4aed-8228-e19549d3e1de" (UID: "ce80464b-ca17-4aed-8228-e19549d3e1de"). InnerVolumeSpecName "kube-api-access-hfhh4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:08.096102 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.096044    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-istiod-ca-cert" pod "ca6dddc7-2d29-4500-ab3f-7c982fc8064d" (UID: "ca6dddc7-2d29-4500-ab3f-7c982fc8064d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:08.096449 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.096318    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f67f5efb-73b5-49fa-b044-36471405ce25-workload-socket" pod "f67f5efb-73b5-49fa-b044-36471405ce25" (UID: "f67f5efb-73b5-49fa-b044-36471405ce25"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.096567 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.096453    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddf16588-cce5-46af-8ff3-e927fb5d76b5-istio-token" pod "ddf16588-cce5-46af-8ff3-e927fb5d76b5" (UID: "ddf16588-cce5-46af-8ff3-e927fb5d76b5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:08.096567 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.096534    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-istio-podinfo" pod "ca6dddc7-2d29-4500-ab3f-7c982fc8064d" (UID: "ca6dddc7-2d29-4500-ab3f-7c982fc8064d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:43:08.096808 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.096788    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce80464b-ca17-4aed-8228-e19549d3e1de-workload-socket" pod "ce80464b-ca17-4aed-8228-e19549d3e1de" (UID: "ce80464b-ca17-4aed-8228-e19549d3e1de"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.097354 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.097069    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f67f5efb-73b5-49fa-b044-36471405ce25-kube-api-access-fjdps" pod "f67f5efb-73b5-49fa-b044-36471405ce25" (UID: "f67f5efb-73b5-49fa-b044-36471405ce25"). InnerVolumeSpecName "kube-api-access-fjdps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:08.097537 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.097513    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ce80464b-ca17-4aed-8228-e19549d3e1de-istio-podinfo" pod "ce80464b-ca17-4aed-8228-e19549d3e1de" (UID: "ce80464b-ca17-4aed-8228-e19549d3e1de"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:43:08.097739 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.097722    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddf16588-cce5-46af-8ff3-e927fb5d76b5-credential-socket" pod "ddf16588-cce5-46af-8ff3-e927fb5d76b5" (UID: "ddf16588-cce5-46af-8ff3-e927fb5d76b5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.098589 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.097582    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f90bc51f-a0ff-47f9-a334-1a2479c933fb-kube-api-access-zfgl5" pod "f90bc51f-a0ff-47f9-a334-1a2479c933fb" (UID: "f90bc51f-a0ff-47f9-a334-1a2479c933fb"). InnerVolumeSpecName "kube-api-access-zfgl5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:08.098982 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.098729    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f67f5efb-73b5-49fa-b044-36471405ce25-istio-data" pod "f67f5efb-73b5-49fa-b044-36471405ce25" (UID: "f67f5efb-73b5-49fa-b044-36471405ce25"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.100547 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.100331    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-kube-api-access-xm9l5" pod "ca6dddc7-2d29-4500-ab3f-7c982fc8064d" (UID: "ca6dddc7-2d29-4500-ab3f-7c982fc8064d"). InnerVolumeSpecName "kube-api-access-xm9l5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:08.100658 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.100633    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d44e37a-fe67-40e5-a3d4-c7665f508b76-istio-token" pod "5d44e37a-fe67-40e5-a3d4-c7665f508b76" (UID: "5d44e37a-fe67-40e5-a3d4-c7665f508b76"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:08.101391 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.100975    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f90bc51f-a0ff-47f9-a334-1a2479c933fb-istio-podinfo" pod "f90bc51f-a0ff-47f9-a334-1a2479c933fb" (UID: "f90bc51f-a0ff-47f9-a334-1a2479c933fb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:43:08.101391 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.101178    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d44e37a-fe67-40e5-a3d4-c7665f508b76-istio-data" pod "5d44e37a-fe67-40e5-a3d4-c7665f508b76" (UID: "5d44e37a-fe67-40e5-a3d4-c7665f508b76"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.102855 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.102629    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f90bc51f-a0ff-47f9-a334-1a2479c933fb-istio-token" pod "f90bc51f-a0ff-47f9-a334-1a2479c933fb" (UID: "f90bc51f-a0ff-47f9-a334-1a2479c933fb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:08.103206 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.103139    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddf16588-cce5-46af-8ff3-e927fb5d76b5-workload-socket" pod "ddf16588-cce5-46af-8ff3-e927fb5d76b5" (UID: "ddf16588-cce5-46af-8ff3-e927fb5d76b5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.103312 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.103216    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f67f5efb-73b5-49fa-b044-36471405ce25-istiod-ca-cert" pod "f67f5efb-73b5-49fa-b044-36471405ce25" (UID: "f67f5efb-73b5-49fa-b044-36471405ce25"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:08.103751 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.103707    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f90bc51f-a0ff-47f9-a334-1a2479c933fb-istio-envoy" pod "f90bc51f-a0ff-47f9-a334-1a2479c933fb" (UID: "f90bc51f-a0ff-47f9-a334-1a2479c933fb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.104110 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.104037    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce80464b-ca17-4aed-8228-e19549d3e1de-istio-token" pod "ce80464b-ca17-4aed-8228-e19549d3e1de" (UID: "ce80464b-ca17-4aed-8228-e19549d3e1de"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:08.104110 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.104054    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ddf16588-cce5-46af-8ff3-e927fb5d76b5-istio-podinfo" pod "ddf16588-cce5-46af-8ff3-e927fb5d76b5" (UID: "ddf16588-cce5-46af-8ff3-e927fb5d76b5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:43:08.104110 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.104079    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d44e37a-fe67-40e5-a3d4-c7665f508b76-istio-envoy" pod "5d44e37a-fe67-40e5-a3d4-c7665f508b76" (UID: "5d44e37a-fe67-40e5-a3d4-c7665f508b76"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.104356 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.104313    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f67f5efb-73b5-49fa-b044-36471405ce25-istio-envoy" pod "f67f5efb-73b5-49fa-b044-36471405ce25" (UID: "f67f5efb-73b5-49fa-b044-36471405ce25"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.105953 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.105795    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddf16588-cce5-46af-8ff3-e927fb5d76b5-istio-envoy" pod "ddf16588-cce5-46af-8ff3-e927fb5d76b5" (UID: "ddf16588-cce5-46af-8ff3-e927fb5d76b5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.149947 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.149239    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"
Oct 30 19:43:08.179874 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.179836    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d44e37a-fe67-40e5-a3d4-c7665f508b76-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180108 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180003    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180108 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180020    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f90bc51f-a0ff-47f9-a334-1a2479c933fb-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180108 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180033    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f67f5efb-73b5-49fa-b044-36471405ce25-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180108 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180045    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddf16588-cce5-46af-8ff3-e927fb5d76b5-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180108 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180057    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d44e37a-fe67-40e5-a3d4-c7665f508b76-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180108 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180068    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f67f5efb-73b5-49fa-b044-36471405ce25-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180108 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180080    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddf16588-cce5-46af-8ff3-e927fb5d76b5-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180108 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180095    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce80464b-ca17-4aed-8228-e19549d3e1de-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180108 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180107    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfhh4\" (UniqueName: \"kubernetes.io/projected/ce80464b-ca17-4aed-8228-e19549d3e1de-kube-api-access-hfhh4\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180126    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f67f5efb-73b5-49fa-b044-36471405ce25-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180143    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f90bc51f-a0ff-47f9-a334-1a2479c933fb-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180157    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f67f5efb-73b5-49fa-b044-36471405ce25-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180170    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f67f5efb-73b5-49fa-b044-36471405ce25-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180183    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d44e37a-fe67-40e5-a3d4-c7665f508b76-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180196    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180210    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f67f5efb-73b5-49fa-b044-36471405ce25-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180224    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce80464b-ca17-4aed-8228-e19549d3e1de-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180238    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfgl5\" (UniqueName: \"kubernetes.io/projected/f90bc51f-a0ff-47f9-a334-1a2479c933fb-kube-api-access-zfgl5\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180253    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d44e37a-fe67-40e5-a3d4-c7665f508b76-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180268    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce80464b-ca17-4aed-8228-e19549d3e1de-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180282    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180296    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180312    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xm9l5\" (UniqueName: \"kubernetes.io/projected/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-kube-api-access-xm9l5\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180325    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddf16588-cce5-46af-8ff3-e927fb5d76b5-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180339    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce80464b-ca17-4aed-8228-e19549d3e1de-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180359    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f90bc51f-a0ff-47f9-a334-1a2479c933fb-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180373    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d44e37a-fe67-40e5-a3d4-c7665f508b76-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180387    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f67f5efb-73b5-49fa-b044-36471405ce25-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180400    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fjdps\" (UniqueName: \"kubernetes.io/projected/f67f5efb-73b5-49fa-b044-36471405ce25-kube-api-access-fjdps\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180417    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f90bc51f-a0ff-47f9-a334-1a2479c933fb-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180433    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddf16588-cce5-46af-8ff3-e927fb5d76b5-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180446    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d44e37a-fe67-40e5-a3d4-c7665f508b76-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180461    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce80464b-ca17-4aed-8228-e19549d3e1de-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180525    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f90bc51f-a0ff-47f9-a334-1a2479c933fb-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180560    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddf16588-cce5-46af-8ff3-e927fb5d76b5-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180575    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180588    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce80464b-ca17-4aed-8228-e19549d3e1de-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180602    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f90bc51f-a0ff-47f9-a334-1a2479c933fb-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180618    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f90bc51f-a0ff-47f9-a334-1a2479c933fb-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180632    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.180638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180645    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f67f5efb-73b5-49fa-b044-36471405ce25-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.182401 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180659    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddf16588-cce5-46af-8ff3-e927fb5d76b5-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.182401 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180672    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d44e37a-fe67-40e5-a3d4-c7665f508b76-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.182401 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180687    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2lb5\" (UniqueName: \"kubernetes.io/projected/ddf16588-cce5-46af-8ff3-e927fb5d76b5-kube-api-access-g2lb5\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.182401 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180701    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f90bc51f-a0ff-47f9-a334-1a2479c933fb-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.182401 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180717    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.182401 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180731    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c54js\" (UniqueName: \"kubernetes.io/projected/5d44e37a-fe67-40e5-a3d4-c7665f508b76-kube-api-access-c54js\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.182401 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180745    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d44e37a-fe67-40e5-a3d4-c7665f508b76-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.182401 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180759    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddf16588-cce5-46af-8ff3-e927fb5d76b5-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.182401 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180772    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddf16588-cce5-46af-8ff3-e927fb5d76b5-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.182401 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180786    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce80464b-ca17-4aed-8228-e19549d3e1de-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.182401 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180800    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca6dddc7-2d29-4500-ab3f-7c982fc8064d-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.182401 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.180814    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce80464b-ca17-4aed-8228-e19549d3e1de-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.274474 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.274425    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-0c305f-5f5c4698b6-kzfnv"]
Oct 30 19:43:08.287679 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.281840    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a16775f-a08c-433c-8707-9eed8d73e7f9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a16775f-a08c-433c-8707-9eed8d73e7f9-credential-socket\") pod \"1a16775f-a08c-433c-8707-9eed8d73e7f9\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") "
Oct 30 19:43:08.287679 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.281976    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a16775f-a08c-433c-8707-9eed8d73e7f9-istio-token\" (UniqueName: \"kubernetes.io/projected/1a16775f-a08c-433c-8707-9eed8d73e7f9-istio-token\") pod \"1a16775f-a08c-433c-8707-9eed8d73e7f9\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") "
Oct 30 19:43:08.287679 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.282032    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a16775f-a08c-433c-8707-9eed8d73e7f9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a16775f-a08c-433c-8707-9eed8d73e7f9-istio-envoy\") pod \"1a16775f-a08c-433c-8707-9eed8d73e7f9\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") "
Oct 30 19:43:08.287679 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.282096    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a16775f-a08c-433c-8707-9eed8d73e7f9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a16775f-a08c-433c-8707-9eed8d73e7f9-workload-socket\") pod \"1a16775f-a08c-433c-8707-9eed8d73e7f9\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") "
Oct 30 19:43:08.287679 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.282146    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1a16775f-a08c-433c-8707-9eed8d73e7f9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a16775f-a08c-433c-8707-9eed8d73e7f9-istiod-ca-cert\") pod \"1a16775f-a08c-433c-8707-9eed8d73e7f9\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") "
Oct 30 19:43:08.287679 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.282237    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1a16775f-a08c-433c-8707-9eed8d73e7f9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a16775f-a08c-433c-8707-9eed8d73e7f9-istio-podinfo\") pod \"1a16775f-a08c-433c-8707-9eed8d73e7f9\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") "
Oct 30 19:43:08.287679 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.282283    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a16775f-a08c-433c-8707-9eed8d73e7f9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a16775f-a08c-433c-8707-9eed8d73e7f9-istio-data\") pod \"1a16775f-a08c-433c-8707-9eed8d73e7f9\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") "
Oct 30 19:43:08.287679 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.282405    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a16775f-a08c-433c-8707-9eed8d73e7f9-kube-api-access-5wsf5\" (UniqueName: \"kubernetes.io/projected/1a16775f-a08c-433c-8707-9eed8d73e7f9-kube-api-access-5wsf5\") pod \"1a16775f-a08c-433c-8707-9eed8d73e7f9\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") "
Oct 30 19:43:08.287679 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.283575    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a16775f-a08c-433c-8707-9eed8d73e7f9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a16775f-a08c-433c-8707-9eed8d73e7f9-workload-certs\") pod \"1a16775f-a08c-433c-8707-9eed8d73e7f9\" (UID: \"1a16775f-a08c-433c-8707-9eed8d73e7f9\") "
Oct 30 19:43:08.287679 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.283959    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a16775f-a08c-433c-8707-9eed8d73e7f9-workload-socket" pod "1a16775f-a08c-433c-8707-9eed8d73e7f9" (UID: "1a16775f-a08c-433c-8707-9eed8d73e7f9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.287679 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.284310    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a16775f-a08c-433c-8707-9eed8d73e7f9-workload-certs" pod "1a16775f-a08c-433c-8707-9eed8d73e7f9" (UID: "1a16775f-a08c-433c-8707-9eed8d73e7f9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.287679 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.284509    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a16775f-a08c-433c-8707-9eed8d73e7f9-credential-socket" pod "1a16775f-a08c-433c-8707-9eed8d73e7f9" (UID: "1a16775f-a08c-433c-8707-9eed8d73e7f9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.287679 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.284671    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a16775f-a08c-433c-8707-9eed8d73e7f9-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.287679 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.284696    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a16775f-a08c-433c-8707-9eed8d73e7f9-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.287679 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.284709    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a16775f-a08c-433c-8707-9eed8d73e7f9-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.287679 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.285047    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a16775f-a08c-433c-8707-9eed8d73e7f9-istiod-ca-cert" pod "1a16775f-a08c-433c-8707-9eed8d73e7f9" (UID: "1a16775f-a08c-433c-8707-9eed8d73e7f9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:08.292669 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.292410    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a16775f-a08c-433c-8707-9eed8d73e7f9-istio-data" pod "1a16775f-a08c-433c-8707-9eed8d73e7f9" (UID: "1a16775f-a08c-433c-8707-9eed8d73e7f9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.301331 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.301297    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1a16775f-a08c-433c-8707-9eed8d73e7f9-istio-podinfo" pod "1a16775f-a08c-433c-8707-9eed8d73e7f9" (UID: "1a16775f-a08c-433c-8707-9eed8d73e7f9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:43:08.301331 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.301319    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a16775f-a08c-433c-8707-9eed8d73e7f9-istio-envoy" pod "1a16775f-a08c-433c-8707-9eed8d73e7f9" (UID: "1a16775f-a08c-433c-8707-9eed8d73e7f9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.302609 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.302582    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a16775f-a08c-433c-8707-9eed8d73e7f9-istio-token" pod "1a16775f-a08c-433c-8707-9eed8d73e7f9" (UID: "1a16775f-a08c-433c-8707-9eed8d73e7f9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:08.302886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.302843    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a16775f-a08c-433c-8707-9eed8d73e7f9-kube-api-access-5wsf5" pod "1a16775f-a08c-433c-8707-9eed8d73e7f9" (UID: "1a16775f-a08c-433c-8707-9eed8d73e7f9"). InnerVolumeSpecName "kube-api-access-5wsf5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:08.350062 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.350041    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"
Oct 30 19:43:08.376382 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.376292    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"
Oct 30 19:43:08.387135 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.386758    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wsf5\" (UniqueName: \"kubernetes.io/projected/1a16775f-a08c-433c-8707-9eed8d73e7f9-kube-api-access-5wsf5\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.387135 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.386788    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a16775f-a08c-433c-8707-9eed8d73e7f9-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.387135 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.386802    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a16775f-a08c-433c-8707-9eed8d73e7f9-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.387135 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.386816    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a16775f-a08c-433c-8707-9eed8d73e7f9-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.387135 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.386828    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a16775f-a08c-433c-8707-9eed8d73e7f9-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.387135 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.386840    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a16775f-a08c-433c-8707-9eed8d73e7f9-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.490622 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.490532    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ed86722-1b31-4348-ac72-136909f507c3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ed86722-1b31-4348-ac72-136909f507c3-workload-socket\") pod \"1ed86722-1b31-4348-ac72-136909f507c3\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") "
Oct 30 19:43:08.491097 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.490632    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ed86722-1b31-4348-ac72-136909f507c3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ed86722-1b31-4348-ac72-136909f507c3-istio-envoy\") pod \"1ed86722-1b31-4348-ac72-136909f507c3\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") "
Oct 30 19:43:08.491097 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.490674    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34a1c006-cd05-48d7-9cb1-00635b1f4729-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34a1c006-cd05-48d7-9cb1-00635b1f4729-credential-socket\") pod \"34a1c006-cd05-48d7-9cb1-00635b1f4729\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") "
Oct 30 19:43:08.491097 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.490746    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34a1c006-cd05-48d7-9cb1-00635b1f4729-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34a1c006-cd05-48d7-9cb1-00635b1f4729-workload-certs\") pod \"34a1c006-cd05-48d7-9cb1-00635b1f4729\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") "
Oct 30 19:43:08.491097 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.490852    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/34a1c006-cd05-48d7-9cb1-00635b1f4729-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34a1c006-cd05-48d7-9cb1-00635b1f4729-istio-podinfo\") pod \"34a1c006-cd05-48d7-9cb1-00635b1f4729\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") "
Oct 30 19:43:08.491097 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.490922    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ed86722-1b31-4348-ac72-136909f507c3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ed86722-1b31-4348-ac72-136909f507c3-workload-certs\") pod \"1ed86722-1b31-4348-ac72-136909f507c3\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") "
Oct 30 19:43:08.491097 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.490967    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1ed86722-1b31-4348-ac72-136909f507c3-kube-api-access-nmd7c\" (UniqueName: \"kubernetes.io/projected/1ed86722-1b31-4348-ac72-136909f507c3-kube-api-access-nmd7c\") pod \"1ed86722-1b31-4348-ac72-136909f507c3\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") "
Oct 30 19:43:08.491097 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.491038    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34a1c006-cd05-48d7-9cb1-00635b1f4729-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34a1c006-cd05-48d7-9cb1-00635b1f4729-workload-socket\") pod \"34a1c006-cd05-48d7-9cb1-00635b1f4729\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") "
Oct 30 19:43:08.491481 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.491124    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/34a1c006-cd05-48d7-9cb1-00635b1f4729-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34a1c006-cd05-48d7-9cb1-00635b1f4729-istiod-ca-cert\") pod \"34a1c006-cd05-48d7-9cb1-00635b1f4729\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") "
Oct 30 19:43:08.491481 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.491166    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/34a1c006-cd05-48d7-9cb1-00635b1f4729-kube-api-access-xx7kh\" (UniqueName: \"kubernetes.io/projected/34a1c006-cd05-48d7-9cb1-00635b1f4729-kube-api-access-xx7kh\") pod \"34a1c006-cd05-48d7-9cb1-00635b1f4729\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") "
Oct 30 19:43:08.491481 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.491213    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ed86722-1b31-4348-ac72-136909f507c3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ed86722-1b31-4348-ac72-136909f507c3-istio-data\") pod \"1ed86722-1b31-4348-ac72-136909f507c3\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") "
Oct 30 19:43:08.491481 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.491320    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1ed86722-1b31-4348-ac72-136909f507c3-istio-token\" (UniqueName: \"kubernetes.io/projected/1ed86722-1b31-4348-ac72-136909f507c3-istio-token\") pod \"1ed86722-1b31-4348-ac72-136909f507c3\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") "
Oct 30 19:43:08.491481 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.491368    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1ed86722-1b31-4348-ac72-136909f507c3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ed86722-1b31-4348-ac72-136909f507c3-istio-podinfo\") pod \"1ed86722-1b31-4348-ac72-136909f507c3\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") "
Oct 30 19:43:08.491481 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.491428    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34a1c006-cd05-48d7-9cb1-00635b1f4729-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34a1c006-cd05-48d7-9cb1-00635b1f4729-istio-data\") pod \"34a1c006-cd05-48d7-9cb1-00635b1f4729\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") "
Oct 30 19:43:08.491783 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.491548    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/34a1c006-cd05-48d7-9cb1-00635b1f4729-istio-token\" (UniqueName: \"kubernetes.io/projected/34a1c006-cd05-48d7-9cb1-00635b1f4729-istio-token\") pod \"34a1c006-cd05-48d7-9cb1-00635b1f4729\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") "
Oct 30 19:43:08.491836 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.491770    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ed86722-1b31-4348-ac72-136909f507c3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ed86722-1b31-4348-ac72-136909f507c3-credential-socket\") pod \"1ed86722-1b31-4348-ac72-136909f507c3\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") "
Oct 30 19:43:08.491911 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.491853    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34a1c006-cd05-48d7-9cb1-00635b1f4729-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34a1c006-cd05-48d7-9cb1-00635b1f4729-istio-envoy\") pod \"34a1c006-cd05-48d7-9cb1-00635b1f4729\" (UID: \"34a1c006-cd05-48d7-9cb1-00635b1f4729\") "
Oct 30 19:43:08.491980 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.491902    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ed86722-1b31-4348-ac72-136909f507c3-credential-socket" pod "1ed86722-1b31-4348-ac72-136909f507c3" (UID: "1ed86722-1b31-4348-ac72-136909f507c3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.491980 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.491943    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1ed86722-1b31-4348-ac72-136909f507c3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ed86722-1b31-4348-ac72-136909f507c3-istiod-ca-cert\") pod \"1ed86722-1b31-4348-ac72-136909f507c3\" (UID: \"1ed86722-1b31-4348-ac72-136909f507c3\") "
Oct 30 19:43:08.492199 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.492170    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ed86722-1b31-4348-ac72-136909f507c3-workload-socket" pod "1ed86722-1b31-4348-ac72-136909f507c3" (UID: "1ed86722-1b31-4348-ac72-136909f507c3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.493199 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.493172    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ed86722-1b31-4348-ac72-136909f507c3-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.493327 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.493203    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ed86722-1b31-4348-ac72-136909f507c3-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.493561 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.493526    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ed86722-1b31-4348-ac72-136909f507c3-istiod-ca-cert" pod "1ed86722-1b31-4348-ac72-136909f507c3" (UID: "1ed86722-1b31-4348-ac72-136909f507c3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:08.493561 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.493521    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34a1c006-cd05-48d7-9cb1-00635b1f4729-istiod-ca-cert" pod "34a1c006-cd05-48d7-9cb1-00635b1f4729" (UID: "34a1c006-cd05-48d7-9cb1-00635b1f4729"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:08.493853 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.493829    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34a1c006-cd05-48d7-9cb1-00635b1f4729-credential-socket" pod "34a1c006-cd05-48d7-9cb1-00635b1f4729" (UID: "34a1c006-cd05-48d7-9cb1-00635b1f4729"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.494075 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.494039    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34a1c006-cd05-48d7-9cb1-00635b1f4729-workload-certs" pod "34a1c006-cd05-48d7-9cb1-00635b1f4729" (UID: "34a1c006-cd05-48d7-9cb1-00635b1f4729"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.495521 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.495493    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34a1c006-cd05-48d7-9cb1-00635b1f4729-istio-data" pod "34a1c006-cd05-48d7-9cb1-00635b1f4729" (UID: "34a1c006-cd05-48d7-9cb1-00635b1f4729"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.504694 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.504664    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1ed86722-1b31-4348-ac72-136909f507c3-istio-podinfo" pod "1ed86722-1b31-4348-ac72-136909f507c3" (UID: "1ed86722-1b31-4348-ac72-136909f507c3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:43:08.505828 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.505797    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ed86722-1b31-4348-ac72-136909f507c3-workload-certs" pod "1ed86722-1b31-4348-ac72-136909f507c3" (UID: "1ed86722-1b31-4348-ac72-136909f507c3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.506386 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.506358    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ed86722-1b31-4348-ac72-136909f507c3-istio-data" pod "1ed86722-1b31-4348-ac72-136909f507c3" (UID: "1ed86722-1b31-4348-ac72-136909f507c3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.508273 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.508249    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ed86722-1b31-4348-ac72-136909f507c3-kube-api-access-nmd7c" pod "1ed86722-1b31-4348-ac72-136909f507c3" (UID: "1ed86722-1b31-4348-ac72-136909f507c3"). InnerVolumeSpecName "kube-api-access-nmd7c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:08.511439 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.511397    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34a1c006-cd05-48d7-9cb1-00635b1f4729-workload-socket" pod "34a1c006-cd05-48d7-9cb1-00635b1f4729" (UID: "34a1c006-cd05-48d7-9cb1-00635b1f4729"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.513131 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.513078    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ed86722-1b31-4348-ac72-136909f507c3-istio-token" pod "1ed86722-1b31-4348-ac72-136909f507c3" (UID: "1ed86722-1b31-4348-ac72-136909f507c3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:08.513235 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.513142    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34a1c006-cd05-48d7-9cb1-00635b1f4729-kube-api-access-xx7kh" pod "34a1c006-cd05-48d7-9cb1-00635b1f4729" (UID: "34a1c006-cd05-48d7-9cb1-00635b1f4729"). InnerVolumeSpecName "kube-api-access-xx7kh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:08.533228 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.533190    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34a1c006-cd05-48d7-9cb1-00635b1f4729-istio-token" pod "34a1c006-cd05-48d7-9cb1-00635b1f4729" (UID: "34a1c006-cd05-48d7-9cb1-00635b1f4729"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:08.538399 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.538344    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ed86722-1b31-4348-ac72-136909f507c3-istio-envoy" pod "1ed86722-1b31-4348-ac72-136909f507c3" (UID: "1ed86722-1b31-4348-ac72-136909f507c3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.538399 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.538358    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34a1c006-cd05-48d7-9cb1-00635b1f4729-istio-envoy" pod "34a1c006-cd05-48d7-9cb1-00635b1f4729" (UID: "34a1c006-cd05-48d7-9cb1-00635b1f4729"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.542533 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.542503    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/34a1c006-cd05-48d7-9cb1-00635b1f4729-istio-podinfo" pod "34a1c006-cd05-48d7-9cb1-00635b1f4729" (UID: "34a1c006-cd05-48d7-9cb1-00635b1f4729"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:43:08.564556 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.564532    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"
Oct 30 19:43:08.595240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.595191    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34a1c006-cd05-48d7-9cb1-00635b1f4729-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.595443 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.595430    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34a1c006-cd05-48d7-9cb1-00635b1f4729-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.595559 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.595549    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ed86722-1b31-4348-ac72-136909f507c3-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.595646 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.595637    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ed86722-1b31-4348-ac72-136909f507c3-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.595722 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.595712    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34a1c006-cd05-48d7-9cb1-00635b1f4729-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.595816 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.595805    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34a1c006-cd05-48d7-9cb1-00635b1f4729-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.595918 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.595908    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34a1c006-cd05-48d7-9cb1-00635b1f4729-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.596067 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.596054    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ed86722-1b31-4348-ac72-136909f507c3-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.596191 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.596165    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmd7c\" (UniqueName: \"kubernetes.io/projected/1ed86722-1b31-4348-ac72-136909f507c3-kube-api-access-nmd7c\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.596762 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.596744    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34a1c006-cd05-48d7-9cb1-00635b1f4729-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.606976 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.606952    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34a1c006-cd05-48d7-9cb1-00635b1f4729-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.607145 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.607132    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xx7kh\" (UniqueName: \"kubernetes.io/projected/34a1c006-cd05-48d7-9cb1-00635b1f4729-kube-api-access-xx7kh\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.609777 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.609744    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ed86722-1b31-4348-ac72-136909f507c3-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.617534 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.617512    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ed86722-1b31-4348-ac72-136909f507c3-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.617736 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.617717    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ed86722-1b31-4348-ac72-136909f507c3-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.617915 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.617899    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34a1c006-cd05-48d7-9cb1-00635b1f4729-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.623254 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.623221    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0c305f-8677844c-mdcqk" event={"ID":"998e3875-ef86-4381-9d65-87ca3bba0b1d","Type":"ContainerStarted","Data":"5db5e188aa0608047f79609c9bad7a6af1ebabfa5d7589f2444f5089323450c5"}
Oct 30 19:43:08.624691 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.624670    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0c305f-8677844c-mdcqk"
Oct 30 19:43:08.626882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.626821    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt" event={"ID":"46317bf7-b022-49a3-b641-66d47bcfb148","Type":"ContainerDied","Data":"e57aa66b97cbbbaae5f6654fa4131dafd26a858ce79e4d1cd7183a03032744ed"}
Oct 30 19:43:08.627064 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.627048    5907 scope.go:122] "RemoveContainer" containerID="481e1b2abb4671b891b984deeb682056d2be9d662a15fd8161624d1dae7242ec"
Oct 30 19:43:08.627329 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.627314    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"
Oct 30 19:43:08.635165 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.635136    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0c305f-5f5c4698b6-kzfnv" event={"ID":"400f0778-d2b1-4ca0-93ec-59672a71e661","Type":"ContainerStarted","Data":"395a779cd3a65c10bce5023afe914c6b30c87943c446e2536321552706001a20"}
Oct 30 19:43:08.635481 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.635193    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0c305f-5f5c4698b6-kzfnv" event={"ID":"400f0778-d2b1-4ca0-93ec-59672a71e661","Type":"ContainerStarted","Data":"6c610d640e8c2dec303edc6fb0535182038ebd3ebafa97753090aeb148a18017"}
Oct 30 19:43:08.636147 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.636042    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0c305f-5f5c4698b6-kzfnv"
Oct 30 19:43:08.648923 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.647505    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"
Oct 30 19:43:08.648923 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.648058    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc" event={"ID":"f67f5efb-73b5-49fa-b044-36471405ce25","Type":"ContainerDied","Data":"caea4fd122a4dd6aca263d0daa30215d103d8fe1ac9f5c6c64db40f7f8ad2667"}
Oct 30 19:43:08.651415 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.650920    5907 scope.go:122] "RemoveContainer" containerID="fb40a12db00274f0fc33da971134b8ce534acaed2fdd623eef70c8179ab054b3"
Oct 30 19:43:08.654264 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.652644    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s" event={"ID":"ddf16588-cce5-46af-8ff3-e927fb5d76b5","Type":"ContainerDied","Data":"0d04eddde29796b788d4af2955db8df07d933e98d039e6340c309ef877a3861a"}
Oct 30 19:43:08.654264 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.652789    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"
Oct 30 19:43:08.658226 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.658197    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm" event={"ID":"ce80464b-ca17-4aed-8228-e19549d3e1de","Type":"ContainerDied","Data":"ab972325969c5f2dfa535f991749e40dbed83d4a1597d4a6b09ea7ab6d697ed9"}
Oct 30 19:43:08.658472 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.658452    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"
Oct 30 19:43:08.660830 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.660806    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl" event={"ID":"ca6dddc7-2d29-4500-ab3f-7c982fc8064d","Type":"ContainerDied","Data":"097f0af3bec906dfec2d327258313a3ec3a3e5277fdb8a1e41e0873ff208562e"}
Oct 30 19:43:08.661000 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.660967    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"
Oct 30 19:43:08.663616 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.663588    5907 generic.go:356] "Generic (PLEG): container finished" podID="819b9937-3bc3-47ac-9631-b6c4fa30a80d" containerID="e731427a8cd1043751072da437ef799eb543c3b5d60ac0b5602aa9982a7424a9" exitCode=0
Oct 30 19:43:08.663718 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.663629    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"
Oct 30 19:43:08.663718 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.663684    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5" event={"ID":"819b9937-3bc3-47ac-9631-b6c4fa30a80d","Type":"ContainerDied","Data":"e731427a8cd1043751072da437ef799eb543c3b5d60ac0b5602aa9982a7424a9"}
Oct 30 19:43:08.663968 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.663729    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5" event={"ID":"819b9937-3bc3-47ac-9631-b6c4fa30a80d","Type":"ContainerDied","Data":"65243c1dc07173ab9bf3b4ac429cd2a9e8ba70d45c64122882e09eefb2dd0ca2"}
Oct 30 19:43:08.666150 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.666128    5907 generic.go:356] "Generic (PLEG): container finished" podID="34a1c006-cd05-48d7-9cb1-00635b1f4729" containerID="094132593af5cd182cda41d836026dc341d4ec3a843350b75271762cf774f64f" exitCode=0
Oct 30 19:43:08.666271 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.666250    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"
Oct 30 19:43:08.666271 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.666276    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc" event={"ID":"34a1c006-cd05-48d7-9cb1-00635b1f4729","Type":"ContainerDied","Data":"094132593af5cd182cda41d836026dc341d4ec3a843350b75271762cf774f64f"}
Oct 30 19:43:08.666451 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.666311    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc" event={"ID":"34a1c006-cd05-48d7-9cb1-00635b1f4729","Type":"ContainerDied","Data":"82b3736a0f51ffa837c8ce1432735316682b2938a4aee208ad615eacd85f2e3b"}
Oct 30 19:43:08.675355 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.675254    5907 scope.go:122] "RemoveContainer" containerID="581fa8b5cf363fd38edafe510db0b783bead95d822bd1496ca2c678f13e29ec9"
Oct 30 19:43:08.679669 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.679533    5907 generic.go:356] "Generic (PLEG): container finished" podID="1a16775f-a08c-433c-8707-9eed8d73e7f9" containerID="f715990ad4b0265ef5f435e4c064331b3ceff62f6bbf5da249fdca18ea8e8d65" exitCode=0
Oct 30 19:43:08.679669 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.679617    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv" event={"ID":"1a16775f-a08c-433c-8707-9eed8d73e7f9","Type":"ContainerDied","Data":"f715990ad4b0265ef5f435e4c064331b3ceff62f6bbf5da249fdca18ea8e8d65"}
Oct 30 19:43:08.679839 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.679689    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv" event={"ID":"1a16775f-a08c-433c-8707-9eed8d73e7f9","Type":"ContainerDied","Data":"7941f1931ae4b91c3dea442bad1a5800423f8facf73f60377160ebe952ec0b80"}
Oct 30 19:43:08.679839 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.679789    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"
Oct 30 19:43:08.685124 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.685097    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6" event={"ID":"f90bc51f-a0ff-47f9-a334-1a2479c933fb","Type":"ContainerDied","Data":"bb097d21dec36530abe58be0e6b8c0f0cf49663d8d285187ee0d7394645bfc16"}
Oct 30 19:43:08.685310 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.685124    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"
Oct 30 19:43:08.687129 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.687109    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"
Oct 30 19:43:08.692323 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.687112    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2" event={"ID":"5d44e37a-fe67-40e5-a3d4-c7665f508b76","Type":"ContainerDied","Data":"8327b093069ff2b70251d3d40b8b8faf4cf1d3d6750c7431e3297f5533e4bc87"}
Oct 30 19:43:08.692444 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.692334    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7e89da-t5hgg" event={"ID":"713310f5-4d82-4e18-877d-235cfac28327","Type":"ContainerStarted","Data":"ae076d042a9aaba941e1df68930a7d698d4945bd5f02feb795f79659f49c778f"}
Oct 30 19:43:08.699606 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.694726    5907 generic.go:356] "Generic (PLEG): container finished" podID="1ed86722-1b31-4348-ac72-136909f507c3" containerID="1ffc30a47b8cf11e36e1d111813b6890868c79f41730257fa0381f107777e205" exitCode=0
Oct 30 19:43:08.699606 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.694797    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6" event={"ID":"1ed86722-1b31-4348-ac72-136909f507c3","Type":"ContainerDied","Data":"1ffc30a47b8cf11e36e1d111813b6890868c79f41730257fa0381f107777e205"}
Oct 30 19:43:08.699606 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.694842    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6" event={"ID":"1ed86722-1b31-4348-ac72-136909f507c3","Type":"ContainerDied","Data":"55c08be65fb1506a8ca8ab3871af86c22e64cfa5c16dc14cd42f0353b242a8d8"}
Oct 30 19:43:08.699606 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.694985    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"
Oct 30 19:43:08.699606 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.698030    5907 scope.go:122] "RemoveContainer" containerID="9949530bd3505e441f3bd1cf4907e339892d1b7f86b1e6af4a81678edaf82c5b"
Oct 30 19:43:08.719521 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.719491    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/819b9937-3bc3-47ac-9631-b6c4fa30a80d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/819b9937-3bc3-47ac-9631-b6c4fa30a80d-istio-data\") pod \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") "
Oct 30 19:43:08.719644 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.719612    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/819b9937-3bc3-47ac-9631-b6c4fa30a80d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/819b9937-3bc3-47ac-9631-b6c4fa30a80d-istio-podinfo\") pod \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") "
Oct 30 19:43:08.719719 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.719696    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/819b9937-3bc3-47ac-9631-b6c4fa30a80d-istio-token\" (UniqueName: \"kubernetes.io/projected/819b9937-3bc3-47ac-9631-b6c4fa30a80d-istio-token\") pod \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") "
Oct 30 19:43:08.720044 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.719996    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/819b9937-3bc3-47ac-9631-b6c4fa30a80d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/819b9937-3bc3-47ac-9631-b6c4fa30a80d-workload-certs\") pod \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") "
Oct 30 19:43:08.720213 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.720049    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/819b9937-3bc3-47ac-9631-b6c4fa30a80d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/819b9937-3bc3-47ac-9631-b6c4fa30a80d-credential-socket\") pod \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") "
Oct 30 19:43:08.720213 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.720106    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/819b9937-3bc3-47ac-9631-b6c4fa30a80d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/819b9937-3bc3-47ac-9631-b6c4fa30a80d-istiod-ca-cert\") pod \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") "
Oct 30 19:43:08.720213 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.720196    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/819b9937-3bc3-47ac-9631-b6c4fa30a80d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/819b9937-3bc3-47ac-9631-b6c4fa30a80d-workload-socket\") pod \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") "
Oct 30 19:43:08.720721 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.720319    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/819b9937-3bc3-47ac-9631-b6c4fa30a80d-kube-api-access-f42b7\" (UniqueName: \"kubernetes.io/projected/819b9937-3bc3-47ac-9631-b6c4fa30a80d-kube-api-access-f42b7\") pod \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") "
Oct 30 19:43:08.720721 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.720368    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/819b9937-3bc3-47ac-9631-b6c4fa30a80d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/819b9937-3bc3-47ac-9631-b6c4fa30a80d-istio-envoy\") pod \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\" (UID: \"819b9937-3bc3-47ac-9631-b6c4fa30a80d\") "
Oct 30 19:43:08.720721 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.720472    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0c305f-8677844c-mdcqk" podStartSLOduration=2.720455588 podStartE2EDuration="2.720455588s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:43:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:43:08.675093376 +0000 UTC m=+1300.518025317" watchObservedRunningTime="2026-10-30 19:43:08.720455588 +0000 UTC m=+1300.563387528"
Oct 30 19:43:08.722164 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.722142    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/819b9937-3bc3-47ac-9631-b6c4fa30a80d-workload-socket" pod "819b9937-3bc3-47ac-9631-b6c4fa30a80d" (UID: "819b9937-3bc3-47ac-9631-b6c4fa30a80d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.722361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.722339    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/819b9937-3bc3-47ac-9631-b6c4fa30a80d-istio-data" pod "819b9937-3bc3-47ac-9631-b6c4fa30a80d" (UID: "819b9937-3bc3-47ac-9631-b6c4fa30a80d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.722780 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.722758    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/819b9937-3bc3-47ac-9631-b6c4fa30a80d-workload-certs" pod "819b9937-3bc3-47ac-9631-b6c4fa30a80d" (UID: "819b9937-3bc3-47ac-9631-b6c4fa30a80d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.727715 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.727550    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/819b9937-3bc3-47ac-9631-b6c4fa30a80d-credential-socket" pod "819b9937-3bc3-47ac-9631-b6c4fa30a80d" (UID: "819b9937-3bc3-47ac-9631-b6c4fa30a80d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.727805 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.727787    5907 scope.go:122] "RemoveContainer" containerID="1ba2acaa95a720fbb49c7b41d62bf366e9d93025619d73e65606f4c39afa6b91"
Oct 30 19:43:08.728254 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.727851    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/819b9937-3bc3-47ac-9631-b6c4fa30a80d-istiod-ca-cert" pod "819b9937-3bc3-47ac-9631-b6c4fa30a80d" (UID: "819b9937-3bc3-47ac-9631-b6c4fa30a80d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:08.728254 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.727906    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"]
Oct 30 19:43:08.728645 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.728625    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/819b9937-3bc3-47ac-9631-b6c4fa30a80d-istio-podinfo" pod "819b9937-3bc3-47ac-9631-b6c4fa30a80d" (UID: "819b9937-3bc3-47ac-9631-b6c4fa30a80d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:43:08.730920 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.730899    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-dphhc"]
Oct 30 19:43:08.732080 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.732055    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/819b9937-3bc3-47ac-9631-b6c4fa30a80d-istio-envoy" pod "819b9937-3bc3-47ac-9631-b6c4fa30a80d" (UID: "819b9937-3bc3-47ac-9631-b6c4fa30a80d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:08.732733 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.732708    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/819b9937-3bc3-47ac-9631-b6c4fa30a80d-istio-token" pod "819b9937-3bc3-47ac-9631-b6c4fa30a80d" (UID: "819b9937-3bc3-47ac-9631-b6c4fa30a80d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:08.742243 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.742187    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/819b9937-3bc3-47ac-9631-b6c4fa30a80d-kube-api-access-f42b7" pod "819b9937-3bc3-47ac-9631-b6c4fa30a80d" (UID: "819b9937-3bc3-47ac-9631-b6c4fa30a80d"). InnerVolumeSpecName "kube-api-access-f42b7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:08.764051 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.764022    5907 patch_prober.go:29] interesting pod/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:43:08.764164 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.764082    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s" podUID="ddf16588-cce5-46af-8ff3-e927fb5d76b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:43:08.769751 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.769735    5907 scope.go:122] "RemoveContainer" containerID="e731427a8cd1043751072da437ef799eb543c3b5d60ac0b5602aa9982a7424a9"
Oct 30 19:43:08.772107 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.772060    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0c305f-5f5c4698b6-kzfnv" podStartSLOduration=2.772044169 podStartE2EDuration="2.772044169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:43:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:43:08.765962812 +0000 UTC m=+1300.608894751" watchObservedRunningTime="2026-10-30 19:43:08.772044169 +0000 UTC m=+1300.614976110"
Oct 30 19:43:08.828948 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.828917    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/819b9937-3bc3-47ac-9631-b6c4fa30a80d-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.828948 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.828949    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/819b9937-3bc3-47ac-9631-b6c4fa30a80d-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.829166 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.828962    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/819b9937-3bc3-47ac-9631-b6c4fa30a80d-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.829166 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.828978    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/819b9937-3bc3-47ac-9631-b6c4fa30a80d-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.829166 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.828991    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/819b9937-3bc3-47ac-9631-b6c4fa30a80d-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.829166 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.829006    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/819b9937-3bc3-47ac-9631-b6c4fa30a80d-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.829166 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.829019    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/819b9937-3bc3-47ac-9631-b6c4fa30a80d-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.829166 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.829033    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f42b7\" (UniqueName: \"kubernetes.io/projected/819b9937-3bc3-47ac-9631-b6c4fa30a80d-kube-api-access-f42b7\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.829166 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.829047    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/819b9937-3bc3-47ac-9631-b6c4fa30a80d-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:08.830261 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.829746    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"]
Oct 30 19:43:08.834743 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.834718    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-wgkbt"]
Oct 30 19:43:08.870885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.870838    5907 scope.go:122] "RemoveContainer" containerID="e731427a8cd1043751072da437ef799eb543c3b5d60ac0b5602aa9982a7424a9"
Oct 30 19:43:08.871246 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.871229    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"]
Oct 30 19:43:08.871640 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:43:08.871500    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e731427a8cd1043751072da437ef799eb543c3b5d60ac0b5602aa9982a7424a9\": container with ID starting with e731427a8cd1043751072da437ef799eb543c3b5d60ac0b5602aa9982a7424a9 not found: ID does not exist" containerID="e731427a8cd1043751072da437ef799eb543c3b5d60ac0b5602aa9982a7424a9"
Oct 30 19:43:08.871853 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.871808    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e731427a8cd1043751072da437ef799eb543c3b5d60ac0b5602aa9982a7424a9"} err="failed to get container status \"e731427a8cd1043751072da437ef799eb543c3b5d60ac0b5602aa9982a7424a9\": rpc error: code = NotFound desc = could not find container \"e731427a8cd1043751072da437ef799eb543c3b5d60ac0b5602aa9982a7424a9\": container with ID starting with e731427a8cd1043751072da437ef799eb543c3b5d60ac0b5602aa9982a7424a9 not found: ID does not exist"
Oct 30 19:43:08.872006 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.871989    5907 scope.go:122] "RemoveContainer" containerID="094132593af5cd182cda41d836026dc341d4ec3a843350b75271762cf774f64f"
Oct 30 19:43:08.875906 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.875883    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-jmktm"]
Oct 30 19:43:08.896879 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.896839    5907 scope.go:122] "RemoveContainer" containerID="094132593af5cd182cda41d836026dc341d4ec3a843350b75271762cf774f64f"
Oct 30 19:43:08.897782 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:43:08.897753    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"094132593af5cd182cda41d836026dc341d4ec3a843350b75271762cf774f64f\": container with ID starting with 094132593af5cd182cda41d836026dc341d4ec3a843350b75271762cf774f64f not found: ID does not exist" containerID="094132593af5cd182cda41d836026dc341d4ec3a843350b75271762cf774f64f"
Oct 30 19:43:08.898347 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.898317    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"094132593af5cd182cda41d836026dc341d4ec3a843350b75271762cf774f64f"} err="failed to get container status \"094132593af5cd182cda41d836026dc341d4ec3a843350b75271762cf774f64f\": rpc error: code = NotFound desc = could not find container \"094132593af5cd182cda41d836026dc341d4ec3a843350b75271762cf774f64f\": container with ID starting with 094132593af5cd182cda41d836026dc341d4ec3a843350b75271762cf774f64f not found: ID does not exist"
Oct 30 19:43:08.898471 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:08.898458    5907 scope.go:122] "RemoveContainer" containerID="f715990ad4b0265ef5f435e4c064331b3ceff62f6bbf5da249fdca18ea8e8d65"
Oct 30 19:43:09.088943 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:09.061690    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"]
Oct 30 19:43:09.108150 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:09.100468    5907 scope.go:122] "RemoveContainer" containerID="f715990ad4b0265ef5f435e4c064331b3ceff62f6bbf5da249fdca18ea8e8d65"
Oct 30 19:43:09.108985 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:43:09.108465    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f715990ad4b0265ef5f435e4c064331b3ceff62f6bbf5da249fdca18ea8e8d65\": container with ID starting with f715990ad4b0265ef5f435e4c064331b3ceff62f6bbf5da249fdca18ea8e8d65 not found: ID does not exist" containerID="f715990ad4b0265ef5f435e4c064331b3ceff62f6bbf5da249fdca18ea8e8d65"
Oct 30 19:43:09.108985 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:09.108511    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f715990ad4b0265ef5f435e4c064331b3ceff62f6bbf5da249fdca18ea8e8d65"} err="failed to get container status \"f715990ad4b0265ef5f435e4c064331b3ceff62f6bbf5da249fdca18ea8e8d65\": rpc error: code = NotFound desc = could not find container \"f715990ad4b0265ef5f435e4c064331b3ceff62f6bbf5da249fdca18ea8e8d65\": container with ID starting with f715990ad4b0265ef5f435e4c064331b3ceff62f6bbf5da249fdca18ea8e8d65 not found: ID does not exist"
Oct 30 19:43:09.108985 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:09.108541    5907 scope.go:122] "RemoveContainer" containerID="7b9616a4c6ae3b1c8d4fa31c97cbc3ac8b49a47c6e5587696dc44603e63a8907"
Oct 30 19:43:09.160201 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:09.154926    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-lhmvl"]
Oct 30 19:43:09.204234 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:09.179703    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"]
Oct 30 19:43:09.204234 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:09.188445    5907 scope.go:122] "RemoveContainer" containerID="ec97339621ffc11976e39b69fcc5b9876e0d56428d6a2f6ace4f447a1f41a41a"
Oct 30 19:43:09.211000 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:09.210959    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-fvrsc"]
Oct 30 19:43:09.218211 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:09.218181    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"]
Oct 30 19:43:09.233686 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:09.233645    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-dd23d1-openshift-default-8f47ff88c-9n87s"]
Oct 30 19:43:09.248091 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:09.247698    5907 scope.go:122] "RemoveContainer" containerID="1ffc30a47b8cf11e36e1d111813b6890868c79f41730257fa0381f107777e205"
Oct 30 19:43:09.258561 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:09.258525    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"]
Oct 30 19:43:09.265840 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:09.265802    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-jvgbv"]
Oct 30 19:43:09.271678 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:09.271650    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"]
Oct 30 19:43:09.274272 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:09.274254    5907 scope.go:122] "RemoveContainer" containerID="1ffc30a47b8cf11e36e1d111813b6890868c79f41730257fa0381f107777e205"
Oct 30 19:43:09.274588 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:43:09.274567    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ffc30a47b8cf11e36e1d111813b6890868c79f41730257fa0381f107777e205\": container with ID starting with 1ffc30a47b8cf11e36e1d111813b6890868c79f41730257fa0381f107777e205 not found: ID does not exist" containerID="1ffc30a47b8cf11e36e1d111813b6890868c79f41730257fa0381f107777e205"
Oct 30 19:43:09.274663 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:09.274595    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ffc30a47b8cf11e36e1d111813b6890868c79f41730257fa0381f107777e205"} err="failed to get container status \"1ffc30a47b8cf11e36e1d111813b6890868c79f41730257fa0381f107777e205\": rpc error: code = NotFound desc = could not find container \"1ffc30a47b8cf11e36e1d111813b6890868c79f41730257fa0381f107777e205\": container with ID starting with 1ffc30a47b8cf11e36e1d111813b6890868c79f41730257fa0381f107777e205 not found: ID does not exist"
Oct 30 19:43:09.276721 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:09.276696    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-fbtx2"]
Oct 30 19:43:09.294925 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:09.294880    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7e89da-t5hgg" podStartSLOduration=3.294853528 podStartE2EDuration="3.294853528s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:43:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:43:09.294286375 +0000 UTC m=+1301.137218314" watchObservedRunningTime="2026-10-30 19:43:09.294853528 +0000 UTC m=+1301.137785464"
Oct 30 19:43:09.337055 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:09.337023    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"]
Oct 30 19:43:09.341583 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:09.341514    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-ncxm6"]
Oct 30 19:43:09.361133 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:09.361100    5907 patch_prober.go:29] interesting pod/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:43:09.361288 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:09.361176    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6" podUID="1ed86722-1b31-4348-ac72-136909f507c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:43:09.383497 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:09.383462    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"]
Oct 30 19:43:09.392771 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:09.392740    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-vmxq6"]
Oct 30 19:43:09.431407 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:09.431377    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"]
Oct 30 19:43:09.437778 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:09.437740    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b4c34-openshift-default-7bfcb4b557-cj7r5"]
Oct 30 19:43:09.713242 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:09.713212    5907 generic.go:356] "Generic (PLEG): container finished" podID="713310f5-4d82-4e18-877d-235cfac28327" containerID="ae076d042a9aaba941e1df68930a7d698d4945bd5f02feb795f79659f49c778f" exitCode=0
Oct 30 19:43:09.713706 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:09.713296    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7e89da-t5hgg" event={"ID":"713310f5-4d82-4e18-877d-235cfac28327","Type":"ContainerDied","Data":"ae076d042a9aaba941e1df68930a7d698d4945bd5f02feb795f79659f49c778f"}
Oct 30 19:43:10.292734 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.292700    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1a16775f-a08c-433c-8707-9eed8d73e7f9" path="/var/lib/kubelet/pods/1a16775f-a08c-433c-8707-9eed8d73e7f9/volumes"
Oct 30 19:43:10.293499 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.293480    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1ed86722-1b31-4348-ac72-136909f507c3" path="/var/lib/kubelet/pods/1ed86722-1b31-4348-ac72-136909f507c3/volumes"
Oct 30 19:43:10.294245 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.294224    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="34a1c006-cd05-48d7-9cb1-00635b1f4729" path="/var/lib/kubelet/pods/34a1c006-cd05-48d7-9cb1-00635b1f4729/volumes"
Oct 30 19:43:10.295442 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.295421    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="46317bf7-b022-49a3-b641-66d47bcfb148" path="/var/lib/kubelet/pods/46317bf7-b022-49a3-b641-66d47bcfb148/volumes"
Oct 30 19:43:10.296127 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.296108    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5d44e37a-fe67-40e5-a3d4-c7665f508b76" path="/var/lib/kubelet/pods/5d44e37a-fe67-40e5-a3d4-c7665f508b76/volumes"
Oct 30 19:43:10.296764 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.296746    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="819b9937-3bc3-47ac-9631-b6c4fa30a80d" path="/var/lib/kubelet/pods/819b9937-3bc3-47ac-9631-b6c4fa30a80d/volumes"
Oct 30 19:43:10.297853 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.297833    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ca6dddc7-2d29-4500-ab3f-7c982fc8064d" path="/var/lib/kubelet/pods/ca6dddc7-2d29-4500-ab3f-7c982fc8064d/volumes"
Oct 30 19:43:10.298594 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.298575    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ce80464b-ca17-4aed-8228-e19549d3e1de" path="/var/lib/kubelet/pods/ce80464b-ca17-4aed-8228-e19549d3e1de/volumes"
Oct 30 19:43:10.299645 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.299622    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ddf16588-cce5-46af-8ff3-e927fb5d76b5" path="/var/lib/kubelet/pods/ddf16588-cce5-46af-8ff3-e927fb5d76b5/volumes"
Oct 30 19:43:10.300304 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.300282    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f67f5efb-73b5-49fa-b044-36471405ce25" path="/var/lib/kubelet/pods/f67f5efb-73b5-49fa-b044-36471405ce25/volumes"
Oct 30 19:43:10.301210 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.301188    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f90bc51f-a0ff-47f9-a334-1a2479c933fb" path="/var/lib/kubelet/pods/f90bc51f-a0ff-47f9-a334-1a2479c933fb/volumes"
Oct 30 19:43:10.463453 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.463428    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:43:10.528504 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.528464    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-istio-podinfo\") pod \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") "
Oct 30 19:43:10.528697 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.528606    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-istio-envoy\") pod \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") "
Oct 30 19:43:10.528697 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.528638    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-istio-data\") pod \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") "
Oct 30 19:43:10.528851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.528827    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-workload-certs\") pod \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") "
Oct 30 19:43:10.529022 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.529002    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-istiod-ca-cert\") pod \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") "
Oct 30 19:43:10.529617 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.529580    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-workload-certs" pod "76a6fce4-3ba8-434a-b126-8a1a8465b1bf" (UID: "76a6fce4-3ba8-434a-b126-8a1a8465b1bf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:10.529748 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.529723    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-istio-data" pod "76a6fce4-3ba8-434a-b126-8a1a8465b1bf" (UID: "76a6fce4-3ba8-434a-b126-8a1a8465b1bf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:10.529966 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.529941    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-istiod-ca-cert" pod "76a6fce4-3ba8-434a-b126-8a1a8465b1bf" (UID: "76a6fce4-3ba8-434a-b126-8a1a8465b1bf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:10.541495 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.529049    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-kube-api-access-fvsz2\" (UniqueName: \"kubernetes.io/projected/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-kube-api-access-fvsz2\") pod \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") "
Oct 30 19:43:10.541615 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.541529    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-credential-socket\") pod \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") "
Oct 30 19:43:10.541671 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.541641    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-workload-socket\") pod \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") "
Oct 30 19:43:10.541737 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.541719    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-istio-token\" (UniqueName: \"kubernetes.io/projected/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-istio-token\") pod \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\" (UID: \"76a6fce4-3ba8-434a-b126-8a1a8465b1bf\") "
Oct 30 19:43:10.542589 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.542563    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:10.542714 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.542594    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:10.542714 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.542598    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-credential-socket" pod "76a6fce4-3ba8-434a-b126-8a1a8465b1bf" (UID: "76a6fce4-3ba8-434a-b126-8a1a8465b1bf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:10.542714 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.542608    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:10.542930 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.542779    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-workload-socket" pod "76a6fce4-3ba8-434a-b126-8a1a8465b1bf" (UID: "76a6fce4-3ba8-434a-b126-8a1a8465b1bf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:10.547350 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.547322    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-istio-podinfo" pod "76a6fce4-3ba8-434a-b126-8a1a8465b1bf" (UID: "76a6fce4-3ba8-434a-b126-8a1a8465b1bf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:43:10.547465 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.547429    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-istio-token" pod "76a6fce4-3ba8-434a-b126-8a1a8465b1bf" (UID: "76a6fce4-3ba8-434a-b126-8a1a8465b1bf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:10.547569 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.547546    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-istio-envoy" pod "76a6fce4-3ba8-434a-b126-8a1a8465b1bf" (UID: "76a6fce4-3ba8-434a-b126-8a1a8465b1bf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:10.559398 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.559371    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-kube-api-access-fvsz2" pod "76a6fce4-3ba8-434a-b126-8a1a8465b1bf" (UID: "76a6fce4-3ba8-434a-b126-8a1a8465b1bf"). InnerVolumeSpecName "kube-api-access-fvsz2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:10.644620 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.644587    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvsz2\" (UniqueName: \"kubernetes.io/projected/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-kube-api-access-fvsz2\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:10.644620 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.644617    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:10.644620 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.644627    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:10.644850 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.644636    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:10.644850 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.644644    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:10.644850 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.644652    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76a6fce4-3ba8-434a-b126-8a1a8465b1bf-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:10.728165 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.728082    5907 generic.go:356] "Generic (PLEG): container finished" podID="76a6fce4-3ba8-434a-b126-8a1a8465b1bf" containerID="c298eb1f658ad6239263f90a8ee68905b2504b8c78bfb5a34b3fe987eeb766ea" exitCode=0
Oct 30 19:43:10.728165 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.728135    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"
Oct 30 19:43:10.728653 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.728163    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6" event={"ID":"76a6fce4-3ba8-434a-b126-8a1a8465b1bf","Type":"ContainerDied","Data":"c298eb1f658ad6239263f90a8ee68905b2504b8c78bfb5a34b3fe987eeb766ea"}
Oct 30 19:43:10.728653 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.728221    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6" event={"ID":"76a6fce4-3ba8-434a-b126-8a1a8465b1bf","Type":"ContainerDied","Data":"cf0ac265c923114d51c593ba27671f6c0071a1b96c506ecd6849b0c276b3dd4c"}
Oct 30 19:43:10.728653 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.728250    5907 scope.go:122] "RemoveContainer" containerID="c298eb1f658ad6239263f90a8ee68905b2504b8c78bfb5a34b3fe987eeb766ea"
Oct 30 19:43:10.741827 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.741804    5907 scope.go:122] "RemoveContainer" containerID="00f49ab6f04e5d256b9f1fd0b6d2d75b2448c8aab2a7c1828d2e8c119788b7fb"
Oct 30 19:43:10.754550 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.754527    5907 scope.go:122] "RemoveContainer" containerID="c298eb1f658ad6239263f90a8ee68905b2504b8c78bfb5a34b3fe987eeb766ea"
Oct 30 19:43:10.754822 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:43:10.754805    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c298eb1f658ad6239263f90a8ee68905b2504b8c78bfb5a34b3fe987eeb766ea\": container with ID starting with c298eb1f658ad6239263f90a8ee68905b2504b8c78bfb5a34b3fe987eeb766ea not found: ID does not exist" containerID="c298eb1f658ad6239263f90a8ee68905b2504b8c78bfb5a34b3fe987eeb766ea"
Oct 30 19:43:10.754902 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.754831    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c298eb1f658ad6239263f90a8ee68905b2504b8c78bfb5a34b3fe987eeb766ea"} err="failed to get container status \"c298eb1f658ad6239263f90a8ee68905b2504b8c78bfb5a34b3fe987eeb766ea\": rpc error: code = NotFound desc = could not find container \"c298eb1f658ad6239263f90a8ee68905b2504b8c78bfb5a34b3fe987eeb766ea\": container with ID starting with c298eb1f658ad6239263f90a8ee68905b2504b8c78bfb5a34b3fe987eeb766ea not found: ID does not exist"
Oct 30 19:43:10.754902 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.754847    5907 scope.go:122] "RemoveContainer" containerID="00f49ab6f04e5d256b9f1fd0b6d2d75b2448c8aab2a7c1828d2e8c119788b7fb"
Oct 30 19:43:10.755116 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:43:10.755100    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00f49ab6f04e5d256b9f1fd0b6d2d75b2448c8aab2a7c1828d2e8c119788b7fb\": container with ID starting with 00f49ab6f04e5d256b9f1fd0b6d2d75b2448c8aab2a7c1828d2e8c119788b7fb not found: ID does not exist" containerID="00f49ab6f04e5d256b9f1fd0b6d2d75b2448c8aab2a7c1828d2e8c119788b7fb"
Oct 30 19:43:10.755156 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.755119    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00f49ab6f04e5d256b9f1fd0b6d2d75b2448c8aab2a7c1828d2e8c119788b7fb"} err="failed to get container status \"00f49ab6f04e5d256b9f1fd0b6d2d75b2448c8aab2a7c1828d2e8c119788b7fb\": rpc error: code = NotFound desc = could not find container \"00f49ab6f04e5d256b9f1fd0b6d2d75b2448c8aab2a7c1828d2e8c119788b7fb\": container with ID starting with 00f49ab6f04e5d256b9f1fd0b6d2d75b2448c8aab2a7c1828d2e8c119788b7fb not found: ID does not exist"
Oct 30 19:43:10.788591 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.788561    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"]
Oct 30 19:43:10.806886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.798424    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6"]
Oct 30 19:43:10.877672 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.877646    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7e89da-t5hgg"
Oct 30 19:43:10.949132 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.949098    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/713310f5-4d82-4e18-877d-235cfac28327-kube-api-access-d26qs\" (UniqueName: \"kubernetes.io/projected/713310f5-4d82-4e18-877d-235cfac28327-kube-api-access-d26qs\") pod \"713310f5-4d82-4e18-877d-235cfac28327\" (UID: \"713310f5-4d82-4e18-877d-235cfac28327\") "
Oct 30 19:43:10.949319 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.949187    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/713310f5-4d82-4e18-877d-235cfac28327-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/713310f5-4d82-4e18-877d-235cfac28327-maas-api-openshift-ca\") pod \"713310f5-4d82-4e18-877d-235cfac28327\" (UID: \"713310f5-4d82-4e18-877d-235cfac28327\") "
Oct 30 19:43:10.949497 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.949465    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/713310f5-4d82-4e18-877d-235cfac28327-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/713310f5-4d82-4e18-877d-235cfac28327-maas-api-xks-ca\") pod \"713310f5-4d82-4e18-877d-235cfac28327\" (UID: \"713310f5-4d82-4e18-877d-235cfac28327\") "
Oct 30 19:43:10.949638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.949577    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/713310f5-4d82-4e18-877d-235cfac28327-maas-api-openshift-ca" pod "713310f5-4d82-4e18-877d-235cfac28327" (UID: "713310f5-4d82-4e18-877d-235cfac28327"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:10.950172 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.950147    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/713310f5-4d82-4e18-877d-235cfac28327-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:10.951528 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.951507    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/713310f5-4d82-4e18-877d-235cfac28327-kube-api-access-d26qs" pod "713310f5-4d82-4e18-877d-235cfac28327" (UID: "713310f5-4d82-4e18-877d-235cfac28327"). InnerVolumeSpecName "kube-api-access-d26qs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:10.952056 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:10.952039    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/713310f5-4d82-4e18-877d-235cfac28327-maas-api-xks-ca" pod "713310f5-4d82-4e18-877d-235cfac28327" (UID: "713310f5-4d82-4e18-877d-235cfac28327"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:43:11.051919 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:11.051822    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/713310f5-4d82-4e18-877d-235cfac28327-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:11.051919 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:11.051850    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d26qs\" (UniqueName: \"kubernetes.io/projected/713310f5-4d82-4e18-877d-235cfac28327-kube-api-access-d26qs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:11.404013 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:11.403977    5907 patch_prober.go:29] interesting pod/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.165:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:43:11.404202 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:11.404032    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-34bdc0-openshift-default-bb6d789bf-vr6l6" podUID="76a6fce4-3ba8-434a-b126-8a1a8465b1bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.165:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:43:11.741363 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:11.741279    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7e89da-t5hgg"
Oct 30 19:43:11.741774 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:11.741276    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7e89da-t5hgg" event={"ID":"713310f5-4d82-4e18-877d-235cfac28327","Type":"ContainerDied","Data":"e3b9f627a3504c7d1ed1ee4b582cc45431bb280984ea0366a26b5ccc1f2f360f"}
Oct 30 19:43:11.741774 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:11.741399    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e3b9f627a3504c7d1ed1ee4b582cc45431bb280984ea0366a26b5ccc1f2f360f"
Oct 30 19:43:12.299370 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.299325    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="76a6fce4-3ba8-434a-b126-8a1a8465b1bf" path="/var/lib/kubelet/pods/76a6fce4-3ba8-434a-b126-8a1a8465b1bf/volumes"
Oct 30 19:43:12.994923 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.994842    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k"]
Oct 30 19:43:12.995835 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.995805    5907 cpu_manager.go:414] "containerMap: removing container" podUID="ca6dddc7-2d29-4500-ab3f-7c982fc8064d" containerName="istio-proxy"
Oct 30 19:43:12.995835 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.995826    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca6dddc7-2d29-4500-ab3f-7c982fc8064d" containerName="istio-proxy"
Oct 30 19:43:12.996017 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.995842    5907 cpu_manager.go:414] "containerMap: removing container" podUID="46317bf7-b022-49a3-b641-66d47bcfb148" containerName="istio-proxy"
Oct 30 19:43:12.996017 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.995847    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="46317bf7-b022-49a3-b641-66d47bcfb148" containerName="istio-proxy"
Oct 30 19:43:12.996017 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.995874    5907 cpu_manager.go:414] "containerMap: removing container" podUID="76a6fce4-3ba8-434a-b126-8a1a8465b1bf" containerName="istio-proxy"
Oct 30 19:43:12.996017 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.995883    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="76a6fce4-3ba8-434a-b126-8a1a8465b1bf" containerName="istio-proxy"
Oct 30 19:43:12.996017 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.995895    5907 cpu_manager.go:414] "containerMap: removing container" podUID="819b9937-3bc3-47ac-9631-b6c4fa30a80d" containerName="istio-proxy"
Oct 30 19:43:12.996017 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.995899    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="819b9937-3bc3-47ac-9631-b6c4fa30a80d" containerName="istio-proxy"
Oct 30 19:43:12.996017 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.995907    5907 cpu_manager.go:414] "containerMap: removing container" podUID="f90bc51f-a0ff-47f9-a334-1a2479c933fb" containerName="istio-proxy"
Oct 30 19:43:12.996017 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.995912    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f90bc51f-a0ff-47f9-a334-1a2479c933fb" containerName="istio-proxy"
Oct 30 19:43:12.996017 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.995936    5907 cpu_manager.go:414] "containerMap: removing container" podUID="ce80464b-ca17-4aed-8228-e19549d3e1de" containerName="istio-proxy"
Oct 30 19:43:12.996017 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.995943    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ce80464b-ca17-4aed-8228-e19549d3e1de" containerName="istio-proxy"
Oct 30 19:43:12.996017 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.995963    5907 cpu_manager.go:414] "containerMap: removing container" podUID="34a1c006-cd05-48d7-9cb1-00635b1f4729" containerName="istio-proxy"
Oct 30 19:43:12.996017 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.995970    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="34a1c006-cd05-48d7-9cb1-00635b1f4729" containerName="istio-proxy"
Oct 30 19:43:12.996017 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.995978    5907 cpu_manager.go:414] "containerMap: removing container" podUID="f67f5efb-73b5-49fa-b044-36471405ce25" containerName="istio-proxy"
Oct 30 19:43:12.996017 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.995983    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f67f5efb-73b5-49fa-b044-36471405ce25" containerName="istio-proxy"
Oct 30 19:43:12.996017 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.995991    5907 cpu_manager.go:414] "containerMap: removing container" podUID="ca6dddc7-2d29-4500-ab3f-7c982fc8064d" containerName="istio-proxy"
Oct 30 19:43:12.996017 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.995997    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca6dddc7-2d29-4500-ab3f-7c982fc8064d" containerName="istio-proxy"
Oct 30 19:43:12.996017 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.996006    5907 cpu_manager.go:414] "containerMap: removing container" podUID="ddf16588-cce5-46af-8ff3-e927fb5d76b5" containerName="istio-proxy"
Oct 30 19:43:12.996017 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.996011    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ddf16588-cce5-46af-8ff3-e927fb5d76b5" containerName="istio-proxy"
Oct 30 19:43:12.996017 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.996023    5907 cpu_manager.go:414] "containerMap: removing container" podUID="5d44e37a-fe67-40e5-a3d4-c7665f508b76" containerName="istio-proxy"
Oct 30 19:43:12.996614 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.996031    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5d44e37a-fe67-40e5-a3d4-c7665f508b76" containerName="istio-proxy"
Oct 30 19:43:12.996614 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.996043    5907 cpu_manager.go:414] "containerMap: removing container" podUID="713310f5-4d82-4e18-877d-235cfac28327" containerName="revoke-keys"
Oct 30 19:43:12.996614 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.996047    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="713310f5-4d82-4e18-877d-235cfac28327" containerName="revoke-keys"
Oct 30 19:43:12.996614 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.996064    5907 cpu_manager.go:414] "containerMap: removing container" podUID="1ed86722-1b31-4348-ac72-136909f507c3" containerName="istio-proxy"
Oct 30 19:43:12.996614 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.996068    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1ed86722-1b31-4348-ac72-136909f507c3" containerName="istio-proxy"
Oct 30 19:43:12.996614 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.996083    5907 cpu_manager.go:414] "containerMap: removing container" podUID="1a16775f-a08c-433c-8707-9eed8d73e7f9" containerName="istio-proxy"
Oct 30 19:43:12.996614 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.996088    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a16775f-a08c-433c-8707-9eed8d73e7f9" containerName="istio-proxy"
Oct 30 19:43:12.996614 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.996203    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca6dddc7-2d29-4500-ab3f-7c982fc8064d" containerName="istio-proxy"
Oct 30 19:43:12.996614 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.996212    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="ddf16588-cce5-46af-8ff3-e927fb5d76b5" containerName="istio-proxy"
Oct 30 19:43:12.996614 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.996219    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="5d44e37a-fe67-40e5-a3d4-c7665f508b76" containerName="istio-proxy"
Oct 30 19:43:12.996614 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.996226    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="713310f5-4d82-4e18-877d-235cfac28327" containerName="revoke-keys"
Oct 30 19:43:12.996614 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.996234    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="1ed86722-1b31-4348-ac72-136909f507c3" containerName="istio-proxy"
Oct 30 19:43:12.996614 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.996242    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a16775f-a08c-433c-8707-9eed8d73e7f9" containerName="istio-proxy"
Oct 30 19:43:12.996614 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.996248    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca6dddc7-2d29-4500-ab3f-7c982fc8064d" containerName="istio-proxy"
Oct 30 19:43:12.996614 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.996257    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="46317bf7-b022-49a3-b641-66d47bcfb148" containerName="istio-proxy"
Oct 30 19:43:12.996614 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.996265    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="76a6fce4-3ba8-434a-b126-8a1a8465b1bf" containerName="istio-proxy"
Oct 30 19:43:12.996614 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.996273    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="819b9937-3bc3-47ac-9631-b6c4fa30a80d" containerName="istio-proxy"
Oct 30 19:43:12.996614 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.996283    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="f90bc51f-a0ff-47f9-a334-1a2479c933fb" containerName="istio-proxy"
Oct 30 19:43:12.996614 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.996290    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="ce80464b-ca17-4aed-8228-e19549d3e1de" containerName="istio-proxy"
Oct 30 19:43:12.996614 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.996295    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="34a1c006-cd05-48d7-9cb1-00635b1f4729" containerName="istio-proxy"
Oct 30 19:43:12.996614 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:12.996300    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="f67f5efb-73b5-49fa-b044-36471405ce25" containerName="istio-proxy"
Oct 30 19:43:13.001504 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:13.001482    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k"
Oct 30 19:43:13.033182 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:13.033152    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k"]
Oct 30 19:43:13.038093 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:13.038066    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-086f2a-serving-cert\""
Oct 30 19:43:13.038093 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:13.038082    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 19:43:13.038293 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:13.038162    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-g97rg\""
Oct 30 19:43:13.083900 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:13.083843    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f957a7e-8e0a-477b-a4da-0d098f7ab5c2-metrics-certs\") pod \"maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k\" (UID: \"3f957a7e-8e0a-477b-a4da-0d098f7ab5c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k"
Oct 30 19:43:13.084139 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:13.084108    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3f957a7e-8e0a-477b-a4da-0d098f7ab5c2-maas-api-tls\") pod \"maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k\" (UID: \"3f957a7e-8e0a-477b-a4da-0d098f7ab5c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k"
Oct 30 19:43:13.084319 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:13.084302    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6h7s\" (UniqueName: \"kubernetes.io/projected/3f957a7e-8e0a-477b-a4da-0d098f7ab5c2-kube-api-access-z6h7s\") pod \"maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k\" (UID: \"3f957a7e-8e0a-477b-a4da-0d098f7ab5c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k"
Oct 30 19:43:13.084379 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:13.084336    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f957a7e-8e0a-477b-a4da-0d098f7ab5c2-openshift-service-ca\") pod \"maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k\" (UID: \"3f957a7e-8e0a-477b-a4da-0d098f7ab5c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k"
Oct 30 19:43:13.193947 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:13.193025    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6h7s\" (UniqueName: \"kubernetes.io/projected/3f957a7e-8e0a-477b-a4da-0d098f7ab5c2-kube-api-access-z6h7s\") pod \"maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k\" (UID: \"3f957a7e-8e0a-477b-a4da-0d098f7ab5c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k"
Oct 30 19:43:13.193947 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:13.193099    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f957a7e-8e0a-477b-a4da-0d098f7ab5c2-openshift-service-ca\") pod \"maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k\" (UID: \"3f957a7e-8e0a-477b-a4da-0d098f7ab5c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k"
Oct 30 19:43:13.193947 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:13.193235    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f957a7e-8e0a-477b-a4da-0d098f7ab5c2-metrics-certs\") pod \"maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k\" (UID: \"3f957a7e-8e0a-477b-a4da-0d098f7ab5c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k"
Oct 30 19:43:13.193947 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:13.193394    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3f957a7e-8e0a-477b-a4da-0d098f7ab5c2-maas-api-tls\") pod \"maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k\" (UID: \"3f957a7e-8e0a-477b-a4da-0d098f7ab5c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k"
Oct 30 19:43:13.205885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:13.196073    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f957a7e-8e0a-477b-a4da-0d098f7ab5c2-openshift-service-ca\") pod \"maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k\" (UID: \"3f957a7e-8e0a-477b-a4da-0d098f7ab5c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k"
Oct 30 19:43:13.250417 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:13.241310    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f957a7e-8e0a-477b-a4da-0d098f7ab5c2-metrics-certs\") pod \"maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k\" (UID: \"3f957a7e-8e0a-477b-a4da-0d098f7ab5c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k"
Oct 30 19:43:13.250417 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:13.241620    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3f957a7e-8e0a-477b-a4da-0d098f7ab5c2-maas-api-tls\") pod \"maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k\" (UID: \"3f957a7e-8e0a-477b-a4da-0d098f7ab5c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k"
Oct 30 19:43:13.260579 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:13.257371    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6h7s\" (UniqueName: \"kubernetes.io/projected/3f957a7e-8e0a-477b-a4da-0d098f7ab5c2-kube-api-access-z6h7s\") pod \"maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k\" (UID: \"3f957a7e-8e0a-477b-a4da-0d098f7ab5c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k"
Oct 30 19:43:13.324526 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:13.324488    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k"
Oct 30 19:43:13.664430 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:13.664399    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k"]
Oct 30 19:43:13.665577 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:43:13.665548    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f957a7e_8e0a_477b_a4da_0d098f7ab5c2.slice/crio-f6c49f869194767c28f49e90089ea42f61b3687a8308275ad7b7073bb2d34cb2 WatchSource:0}: Error finding container f6c49f869194767c28f49e90089ea42f61b3687a8308275ad7b7073bb2d34cb2: Status 404 returned error can't find the container with id f6c49f869194767c28f49e90089ea42f61b3687a8308275ad7b7073bb2d34cb2
Oct 30 19:43:13.666976 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:13.666958    5907 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:43:13.790922 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:13.790887    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k" event={"ID":"3f957a7e-8e0a-477b-a4da-0d098f7ab5c2","Type":"ContainerStarted","Data":"f6c49f869194767c28f49e90089ea42f61b3687a8308275ad7b7073bb2d34cb2"}
Oct 30 19:43:15.543206 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:15.540330    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6a544e22-rf86x"]
Oct 30 19:43:15.543206 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:15.541201    5907 cpu_manager.go:414] "containerMap: removing container" podUID="76a6fce4-3ba8-434a-b126-8a1a8465b1bf" containerName="istio-proxy"
Oct 30 19:43:15.543206 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:15.541216    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="76a6fce4-3ba8-434a-b126-8a1a8465b1bf" containerName="istio-proxy"
Oct 30 19:43:15.543206 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:15.541381    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="76a6fce4-3ba8-434a-b126-8a1a8465b1bf" containerName="istio-proxy"
Oct 30 19:43:15.545591 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:15.545567    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a544e22-rf86x"
Oct 30 19:43:15.569741 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:15.569712    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6a544e22-rf86x"]
Oct 30 19:43:15.594424 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:15.593682    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:43:15.594424 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:15.593726    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:43:15.594424 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:15.594309    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2aa211ac-280d-4294-8f9d-807318fafceb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6a544e22-rf86x\" (UID: \"2aa211ac-280d-4294-8f9d-807318fafceb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a544e22-rf86x"
Oct 30 19:43:15.594424 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:15.594388    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlmwx\" (UniqueName: \"kubernetes.io/projected/2aa211ac-280d-4294-8f9d-807318fafceb-kube-api-access-mlmwx\") pod \"maas-api-revoke-sub-6a544e22-rf86x\" (UID: \"2aa211ac-280d-4294-8f9d-807318fafceb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a544e22-rf86x"
Oct 30 19:43:15.594746 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:15.594486    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2aa211ac-280d-4294-8f9d-807318fafceb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6a544e22-rf86x\" (UID: \"2aa211ac-280d-4294-8f9d-807318fafceb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a544e22-rf86x"
Oct 30 19:43:15.695838 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:15.695801    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2aa211ac-280d-4294-8f9d-807318fafceb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6a544e22-rf86x\" (UID: \"2aa211ac-280d-4294-8f9d-807318fafceb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a544e22-rf86x"
Oct 30 19:43:15.696060 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:15.695959    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2aa211ac-280d-4294-8f9d-807318fafceb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6a544e22-rf86x\" (UID: \"2aa211ac-280d-4294-8f9d-807318fafceb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a544e22-rf86x"
Oct 30 19:43:15.696060 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:15.696018    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlmwx\" (UniqueName: \"kubernetes.io/projected/2aa211ac-280d-4294-8f9d-807318fafceb-kube-api-access-mlmwx\") pod \"maas-api-revoke-sub-6a544e22-rf86x\" (UID: \"2aa211ac-280d-4294-8f9d-807318fafceb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a544e22-rf86x"
Oct 30 19:43:15.696920 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:15.696835    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2aa211ac-280d-4294-8f9d-807318fafceb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6a544e22-rf86x\" (UID: \"2aa211ac-280d-4294-8f9d-807318fafceb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a544e22-rf86x"
Oct 30 19:43:15.698614 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:15.698587    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2aa211ac-280d-4294-8f9d-807318fafceb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6a544e22-rf86x\" (UID: \"2aa211ac-280d-4294-8f9d-807318fafceb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a544e22-rf86x"
Oct 30 19:43:15.737870 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:15.737839    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlmwx\" (UniqueName: \"kubernetes.io/projected/2aa211ac-280d-4294-8f9d-807318fafceb-kube-api-access-mlmwx\") pod \"maas-api-revoke-sub-6a544e22-rf86x\" (UID: \"2aa211ac-280d-4294-8f9d-807318fafceb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a544e22-rf86x"
Oct 30 19:43:15.864206 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:15.864176    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a544e22-rf86x"
Oct 30 19:43:16.227702 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:16.227287    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6a544e22-rf86x"]
Oct 30 19:43:16.243754 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:43:16.243716    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2aa211ac_280d_4294_8f9d_807318fafceb.slice/crio-768e61fc0778e373048d7064a7f02fec8c5902651b4bdec88060c6afdb31eec6 WatchSource:0}: Error finding container 768e61fc0778e373048d7064a7f02fec8c5902651b4bdec88060c6afdb31eec6: Status 404 returned error can't find the container with id 768e61fc0778e373048d7064a7f02fec8c5902651b4bdec88060c6afdb31eec6
Oct 30 19:43:16.758526 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:16.758495    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"]
Oct 30 19:43:16.758899 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:16.758695    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb" podUID="283578cb-7837-4d4a-b91f-dee1d0d74322" containerName="istio-proxy" containerID="cri-o://555010fb5775749893e35e327ebf73af2bc1ac9bcd8b0e180889649887577e51" gracePeriod=30
Oct 30 19:43:16.771597 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:16.771386    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"]
Oct 30 19:43:16.771842 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:16.771817    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl" podUID="210e8e86-7046-4152-9a79-8f3c0fcdc900" containerName="istio-proxy" containerID="cri-o://124ee5b9d32da880da7d58796c17736fda5164ae6c1efc6bb1a6bb003f1a4144" gracePeriod=30
Oct 30 19:43:16.860377 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:16.860347    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a544e22-rf86x" event={"ID":"2aa211ac-280d-4294-8f9d-807318fafceb","Type":"ContainerStarted","Data":"97672ebd1e21daf3833a9e603d443d42fa5fd4c7c4ff1b01b8c3f0e7779d0923"}
Oct 30 19:43:16.860554 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:16.860402    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a544e22-rf86x" event={"ID":"2aa211ac-280d-4294-8f9d-807318fafceb","Type":"ContainerStarted","Data":"768e61fc0778e373048d7064a7f02fec8c5902651b4bdec88060c6afdb31eec6"}
Oct 30 19:43:16.865081 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:16.865046    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k" event={"ID":"3f957a7e-8e0a-477b-a4da-0d098f7ab5c2","Type":"ContainerStarted","Data":"cf0c9fe3897c7ecbc6ee1acc471b246d6a45fb9d16f184d87359d897f4d9144d"}
Oct 30 19:43:16.865243 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:16.865141    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k"
Oct 30 19:43:16.957825 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:16.957631    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k" podStartSLOduration=2.544796564 podStartE2EDuration="4.957609305s" totalImagesPullingTime="2.412812741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:43:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:43:16.956509913 +0000 UTC m=+1308.799441853" watchObservedRunningTime="2026-10-30 19:43:16.957609305 +0000 UTC m=+1308.800541238"
Oct 30 19:43:16.965420 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:16.965364    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a544e22-rf86x" podStartSLOduration=1.9653473510000001 podStartE2EDuration="1.965347351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:43:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:43:16.908641997 +0000 UTC m=+1308.751573960" watchObservedRunningTime="2026-10-30 19:43:16.965347351 +0000 UTC m=+1308.808279292"
Oct 30 19:43:17.051139 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:17.050567    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"]
Oct 30 19:43:17.051139 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:17.050770    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j" podUID="0f23e39e-699c-43ea-bf10-9553aee30c15" containerName="istio-proxy" containerID="cri-o://06731322ab687584e6abc41e6bf8450d2a702884bb8d69e9266631981f61d373" gracePeriod=30
Oct 30 19:43:17.768948 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:43:17.758453    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:43:17.878652 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:17.878616    5907 generic.go:356] "Generic (PLEG): container finished" podID="2aa211ac-280d-4294-8f9d-807318fafceb" containerID="97672ebd1e21daf3833a9e603d443d42fa5fd4c7c4ff1b01b8c3f0e7779d0923" exitCode=0
Oct 30 19:43:17.878847 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:17.878710    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a544e22-rf86x" event={"ID":"2aa211ac-280d-4294-8f9d-807318fafceb","Type":"ContainerDied","Data":"97672ebd1e21daf3833a9e603d443d42fa5fd4c7c4ff1b01b8c3f0e7779d0923"}
Oct 30 19:43:19.093060 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:19.093030    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a544e22-rf86x"
Oct 30 19:43:19.262333 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:19.262303    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2aa211ac-280d-4294-8f9d-807318fafceb-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2aa211ac-280d-4294-8f9d-807318fafceb-maas-api-xks-ca\") pod \"2aa211ac-280d-4294-8f9d-807318fafceb\" (UID: \"2aa211ac-280d-4294-8f9d-807318fafceb\") "
Oct 30 19:43:19.262552 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:19.262433    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2aa211ac-280d-4294-8f9d-807318fafceb-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2aa211ac-280d-4294-8f9d-807318fafceb-maas-api-openshift-ca\") pod \"2aa211ac-280d-4294-8f9d-807318fafceb\" (UID: \"2aa211ac-280d-4294-8f9d-807318fafceb\") "
Oct 30 19:43:19.262552 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:19.262485    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2aa211ac-280d-4294-8f9d-807318fafceb-kube-api-access-mlmwx\" (UniqueName: \"kubernetes.io/projected/2aa211ac-280d-4294-8f9d-807318fafceb-kube-api-access-mlmwx\") pod \"2aa211ac-280d-4294-8f9d-807318fafceb\" (UID: \"2aa211ac-280d-4294-8f9d-807318fafceb\") "
Oct 30 19:43:19.262847 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:19.262819    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2aa211ac-280d-4294-8f9d-807318fafceb-maas-api-openshift-ca" pod "2aa211ac-280d-4294-8f9d-807318fafceb" (UID: "2aa211ac-280d-4294-8f9d-807318fafceb"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:19.263381 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:19.263355    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2aa211ac-280d-4294-8f9d-807318fafceb-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:19.264682 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:19.264653    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2aa211ac-280d-4294-8f9d-807318fafceb-maas-api-xks-ca" pod "2aa211ac-280d-4294-8f9d-807318fafceb" (UID: "2aa211ac-280d-4294-8f9d-807318fafceb"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:43:19.264787 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:19.264769    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2aa211ac-280d-4294-8f9d-807318fafceb-kube-api-access-mlmwx" pod "2aa211ac-280d-4294-8f9d-807318fafceb" (UID: "2aa211ac-280d-4294-8f9d-807318fafceb"). InnerVolumeSpecName "kube-api-access-mlmwx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:19.364729 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:19.364693    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2aa211ac-280d-4294-8f9d-807318fafceb-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:19.364729 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:19.364723    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mlmwx\" (UniqueName: \"kubernetes.io/projected/2aa211ac-280d-4294-8f9d-807318fafceb-kube-api-access-mlmwx\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:19.720963 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:19.720933    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0c305f-5f5c4698b6-kzfnv"
Oct 30 19:43:19.911032 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:19.910997    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a544e22-rf86x" event={"ID":"2aa211ac-280d-4294-8f9d-807318fafceb","Type":"ContainerDied","Data":"768e61fc0778e373048d7064a7f02fec8c5902651b4bdec88060c6afdb31eec6"}
Oct 30 19:43:19.911232 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:19.911067    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="768e61fc0778e373048d7064a7f02fec8c5902651b4bdec88060c6afdb31eec6"
Oct 30 19:43:19.911232 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:19.911007    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a544e22-rf86x"
Oct 30 19:43:20.007217 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:20.007135    5907 patch_prober.go:29] interesting pod/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:43:20.007358 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:20.007237    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j" podUID="0f23e39e-699c-43ea-bf10-9553aee30c15" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:43:20.733290 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:20.733257    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0c305f-8677844c-mdcqk"
Oct 30 19:43:21.935169 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:21.935128    5907 generic.go:356] "Generic (PLEG): container finished" podID="210e8e86-7046-4152-9a79-8f3c0fcdc900" containerID="124ee5b9d32da880da7d58796c17736fda5164ae6c1efc6bb1a6bb003f1a4144" exitCode=0
Oct 30 19:43:21.935674 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:21.935196    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl" event={"ID":"210e8e86-7046-4152-9a79-8f3c0fcdc900","Type":"ContainerDied","Data":"124ee5b9d32da880da7d58796c17736fda5164ae6c1efc6bb1a6bb003f1a4144"}
Oct 30 19:43:21.937133 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:21.937108    5907 generic.go:356] "Generic (PLEG): container finished" podID="283578cb-7837-4d4a-b91f-dee1d0d74322" containerID="555010fb5775749893e35e327ebf73af2bc1ac9bcd8b0e180889649887577e51" exitCode=0
Oct 30 19:43:21.937354 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:21.937199    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb" event={"ID":"283578cb-7837-4d4a-b91f-dee1d0d74322","Type":"ContainerDied","Data":"555010fb5775749893e35e327ebf73af2bc1ac9bcd8b0e180889649887577e51"}
Oct 30 19:43:22.159449 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.159425    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"
Oct 30 19:43:22.163378 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.163358    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"
Oct 30 19:43:22.324622 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.324524    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/283578cb-7837-4d4a-b91f-dee1d0d74322-istio-token\" (UniqueName: \"kubernetes.io/projected/283578cb-7837-4d4a-b91f-dee1d0d74322-istio-token\") pod \"283578cb-7837-4d4a-b91f-dee1d0d74322\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") "
Oct 30 19:43:22.324622 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.324586    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/210e8e86-7046-4152-9a79-8f3c0fcdc900-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/210e8e86-7046-4152-9a79-8f3c0fcdc900-istiod-ca-cert\") pod \"210e8e86-7046-4152-9a79-8f3c0fcdc900\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") "
Oct 30 19:43:22.324908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.324650    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/283578cb-7837-4d4a-b91f-dee1d0d74322-kube-api-access-4zqvn\" (UniqueName: \"kubernetes.io/projected/283578cb-7837-4d4a-b91f-dee1d0d74322-kube-api-access-4zqvn\") pod \"283578cb-7837-4d4a-b91f-dee1d0d74322\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") "
Oct 30 19:43:22.324908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.324698    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/210e8e86-7046-4152-9a79-8f3c0fcdc900-istio-token\" (UniqueName: \"kubernetes.io/projected/210e8e86-7046-4152-9a79-8f3c0fcdc900-istio-token\") pod \"210e8e86-7046-4152-9a79-8f3c0fcdc900\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") "
Oct 30 19:43:22.324908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.324774    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/210e8e86-7046-4152-9a79-8f3c0fcdc900-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/210e8e86-7046-4152-9a79-8f3c0fcdc900-credential-socket\") pod \"210e8e86-7046-4152-9a79-8f3c0fcdc900\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") "
Oct 30 19:43:22.324908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.324797    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/283578cb-7837-4d4a-b91f-dee1d0d74322-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/283578cb-7837-4d4a-b91f-dee1d0d74322-credential-socket\") pod \"283578cb-7837-4d4a-b91f-dee1d0d74322\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") "
Oct 30 19:43:22.324908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.324884    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/283578cb-7837-4d4a-b91f-dee1d0d74322-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/283578cb-7837-4d4a-b91f-dee1d0d74322-istio-envoy\") pod \"283578cb-7837-4d4a-b91f-dee1d0d74322\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") "
Oct 30 19:43:22.325144 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.324916    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/283578cb-7837-4d4a-b91f-dee1d0d74322-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/283578cb-7837-4d4a-b91f-dee1d0d74322-istio-data\") pod \"283578cb-7837-4d4a-b91f-dee1d0d74322\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") "
Oct 30 19:43:22.325144 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.324966    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/210e8e86-7046-4152-9a79-8f3c0fcdc900-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/210e8e86-7046-4152-9a79-8f3c0fcdc900-workload-socket\") pod \"210e8e86-7046-4152-9a79-8f3c0fcdc900\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") "
Oct 30 19:43:22.325144 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.324990    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/283578cb-7837-4d4a-b91f-dee1d0d74322-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/283578cb-7837-4d4a-b91f-dee1d0d74322-workload-socket\") pod \"283578cb-7837-4d4a-b91f-dee1d0d74322\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") "
Oct 30 19:43:22.325144 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.325035    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/283578cb-7837-4d4a-b91f-dee1d0d74322-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/283578cb-7837-4d4a-b91f-dee1d0d74322-istiod-ca-cert\") pod \"283578cb-7837-4d4a-b91f-dee1d0d74322\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") "
Oct 30 19:43:22.325144 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.325060    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/210e8e86-7046-4152-9a79-8f3c0fcdc900-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/210e8e86-7046-4152-9a79-8f3c0fcdc900-istio-data\") pod \"210e8e86-7046-4152-9a79-8f3c0fcdc900\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") "
Oct 30 19:43:22.325144 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.325123    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/210e8e86-7046-4152-9a79-8f3c0fcdc900-istiod-ca-cert" pod "210e8e86-7046-4152-9a79-8f3c0fcdc900" (UID: "210e8e86-7046-4152-9a79-8f3c0fcdc900"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:22.325343 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.325153    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/283578cb-7837-4d4a-b91f-dee1d0d74322-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/283578cb-7837-4d4a-b91f-dee1d0d74322-istio-podinfo\") pod \"283578cb-7837-4d4a-b91f-dee1d0d74322\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") "
Oct 30 19:43:22.325343 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.325212    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/210e8e86-7046-4152-9a79-8f3c0fcdc900-kube-api-access-jrcnp\" (UniqueName: \"kubernetes.io/projected/210e8e86-7046-4152-9a79-8f3c0fcdc900-kube-api-access-jrcnp\") pod \"210e8e86-7046-4152-9a79-8f3c0fcdc900\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") "
Oct 30 19:43:22.325343 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.325244    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/210e8e86-7046-4152-9a79-8f3c0fcdc900-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/210e8e86-7046-4152-9a79-8f3c0fcdc900-istio-podinfo\") pod \"210e8e86-7046-4152-9a79-8f3c0fcdc900\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") "
Oct 30 19:43:22.325343 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.325304    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/283578cb-7837-4d4a-b91f-dee1d0d74322-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/283578cb-7837-4d4a-b91f-dee1d0d74322-workload-certs\") pod \"283578cb-7837-4d4a-b91f-dee1d0d74322\" (UID: \"283578cb-7837-4d4a-b91f-dee1d0d74322\") "
Oct 30 19:43:22.325571 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.325346    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/210e8e86-7046-4152-9a79-8f3c0fcdc900-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/210e8e86-7046-4152-9a79-8f3c0fcdc900-workload-certs\") pod \"210e8e86-7046-4152-9a79-8f3c0fcdc900\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") "
Oct 30 19:43:22.325630 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.325593    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/210e8e86-7046-4152-9a79-8f3c0fcdc900-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/210e8e86-7046-4152-9a79-8f3c0fcdc900-istio-envoy\") pod \"210e8e86-7046-4152-9a79-8f3c0fcdc900\" (UID: \"210e8e86-7046-4152-9a79-8f3c0fcdc900\") "
Oct 30 19:43:22.326716 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.326224    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/210e8e86-7046-4152-9a79-8f3c0fcdc900-workload-socket" pod "210e8e86-7046-4152-9a79-8f3c0fcdc900" (UID: "210e8e86-7046-4152-9a79-8f3c0fcdc900"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:22.326716 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.326507    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/210e8e86-7046-4152-9a79-8f3c0fcdc900-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:22.326716 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.326530    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/210e8e86-7046-4152-9a79-8f3c0fcdc900-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:22.327417 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.327089    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/210e8e86-7046-4152-9a79-8f3c0fcdc900-credential-socket" pod "210e8e86-7046-4152-9a79-8f3c0fcdc900" (UID: "210e8e86-7046-4152-9a79-8f3c0fcdc900"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:22.327803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.327571    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/283578cb-7837-4d4a-b91f-dee1d0d74322-istio-data" pod "283578cb-7837-4d4a-b91f-dee1d0d74322" (UID: "283578cb-7837-4d4a-b91f-dee1d0d74322"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:22.328165 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.327993    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/283578cb-7837-4d4a-b91f-dee1d0d74322-istio-token" pod "283578cb-7837-4d4a-b91f-dee1d0d74322" (UID: "283578cb-7837-4d4a-b91f-dee1d0d74322"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:22.328165 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.327993    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/283578cb-7837-4d4a-b91f-dee1d0d74322-workload-socket" pod "283578cb-7837-4d4a-b91f-dee1d0d74322" (UID: "283578cb-7837-4d4a-b91f-dee1d0d74322"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:22.328306 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.328272    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/283578cb-7837-4d4a-b91f-dee1d0d74322-istiod-ca-cert" pod "283578cb-7837-4d4a-b91f-dee1d0d74322" (UID: "283578cb-7837-4d4a-b91f-dee1d0d74322"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:22.329127 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.328845    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/283578cb-7837-4d4a-b91f-dee1d0d74322-workload-certs" pod "283578cb-7837-4d4a-b91f-dee1d0d74322" (UID: "283578cb-7837-4d4a-b91f-dee1d0d74322"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:22.329127 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.328852    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/210e8e86-7046-4152-9a79-8f3c0fcdc900-istio-data" pod "210e8e86-7046-4152-9a79-8f3c0fcdc900" (UID: "210e8e86-7046-4152-9a79-8f3c0fcdc900"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:22.329127 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.328981    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/283578cb-7837-4d4a-b91f-dee1d0d74322-credential-socket" pod "283578cb-7837-4d4a-b91f-dee1d0d74322" (UID: "283578cb-7837-4d4a-b91f-dee1d0d74322"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:22.329127 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.329103    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/210e8e86-7046-4152-9a79-8f3c0fcdc900-workload-certs" pod "210e8e86-7046-4152-9a79-8f3c0fcdc900" (UID: "210e8e86-7046-4152-9a79-8f3c0fcdc900"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:22.329402 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.329133    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/210e8e86-7046-4152-9a79-8f3c0fcdc900-istio-token" pod "210e8e86-7046-4152-9a79-8f3c0fcdc900" (UID: "210e8e86-7046-4152-9a79-8f3c0fcdc900"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:22.329402 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.329317    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/210e8e86-7046-4152-9a79-8f3c0fcdc900-istio-envoy" pod "210e8e86-7046-4152-9a79-8f3c0fcdc900" (UID: "210e8e86-7046-4152-9a79-8f3c0fcdc900"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:22.330764 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.330725    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/283578cb-7837-4d4a-b91f-dee1d0d74322-istio-envoy" pod "283578cb-7837-4d4a-b91f-dee1d0d74322" (UID: "283578cb-7837-4d4a-b91f-dee1d0d74322"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:22.334319 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.334278    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/283578cb-7837-4d4a-b91f-dee1d0d74322-istio-podinfo" pod "283578cb-7837-4d4a-b91f-dee1d0d74322" (UID: "283578cb-7837-4d4a-b91f-dee1d0d74322"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:43:22.334414 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.334319    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/283578cb-7837-4d4a-b91f-dee1d0d74322-kube-api-access-4zqvn" pod "283578cb-7837-4d4a-b91f-dee1d0d74322" (UID: "283578cb-7837-4d4a-b91f-dee1d0d74322"). InnerVolumeSpecName "kube-api-access-4zqvn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:22.334414 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.334381    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/210e8e86-7046-4152-9a79-8f3c0fcdc900-istio-podinfo" pod "210e8e86-7046-4152-9a79-8f3c0fcdc900" (UID: "210e8e86-7046-4152-9a79-8f3c0fcdc900"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:43:22.334414 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.334391    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/210e8e86-7046-4152-9a79-8f3c0fcdc900-kube-api-access-jrcnp" pod "210e8e86-7046-4152-9a79-8f3c0fcdc900" (UID: "210e8e86-7046-4152-9a79-8f3c0fcdc900"). InnerVolumeSpecName "kube-api-access-jrcnp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:22.428545 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.428504    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/283578cb-7837-4d4a-b91f-dee1d0d74322-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:22.428545 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.428536    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/283578cb-7837-4d4a-b91f-dee1d0d74322-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:22.428831 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.428569    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/283578cb-7837-4d4a-b91f-dee1d0d74322-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:22.428831 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.428595    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/283578cb-7837-4d4a-b91f-dee1d0d74322-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:22.428831 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.428618    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/210e8e86-7046-4152-9a79-8f3c0fcdc900-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:22.428831 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.428637    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/283578cb-7837-4d4a-b91f-dee1d0d74322-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:22.428831 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.428653    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrcnp\" (UniqueName: \"kubernetes.io/projected/210e8e86-7046-4152-9a79-8f3c0fcdc900-kube-api-access-jrcnp\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:22.428831 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.428669    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/210e8e86-7046-4152-9a79-8f3c0fcdc900-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:22.428831 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.428681    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/283578cb-7837-4d4a-b91f-dee1d0d74322-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:22.428831 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.428714    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/210e8e86-7046-4152-9a79-8f3c0fcdc900-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:22.428831 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.428736    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/210e8e86-7046-4152-9a79-8f3c0fcdc900-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:22.428831 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.428747    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/283578cb-7837-4d4a-b91f-dee1d0d74322-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:22.428831 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.428760    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zqvn\" (UniqueName: \"kubernetes.io/projected/283578cb-7837-4d4a-b91f-dee1d0d74322-kube-api-access-4zqvn\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:22.428831 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.428786    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/210e8e86-7046-4152-9a79-8f3c0fcdc900-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:22.428831 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.428800    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/210e8e86-7046-4152-9a79-8f3c0fcdc900-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:22.428831 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.428811    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/283578cb-7837-4d4a-b91f-dee1d0d74322-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:22.725239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.725208    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"
Oct 30 19:43:22.834748 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.834711    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0f23e39e-699c-43ea-bf10-9553aee30c15-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f23e39e-699c-43ea-bf10-9553aee30c15-istio-podinfo\") pod \"0f23e39e-699c-43ea-bf10-9553aee30c15\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") "
Oct 30 19:43:22.834940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.834774    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f23e39e-699c-43ea-bf10-9553aee30c15-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f23e39e-699c-43ea-bf10-9553aee30c15-credential-socket\") pod \"0f23e39e-699c-43ea-bf10-9553aee30c15\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") "
Oct 30 19:43:22.834940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.834828    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f23e39e-699c-43ea-bf10-9553aee30c15-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f23e39e-699c-43ea-bf10-9553aee30c15-istio-envoy\") pod \"0f23e39e-699c-43ea-bf10-9553aee30c15\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") "
Oct 30 19:43:22.834940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.834890    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0f23e39e-699c-43ea-bf10-9553aee30c15-istio-token\" (UniqueName: \"kubernetes.io/projected/0f23e39e-699c-43ea-bf10-9553aee30c15-istio-token\") pod \"0f23e39e-699c-43ea-bf10-9553aee30c15\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") "
Oct 30 19:43:22.835054 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.834945    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f23e39e-699c-43ea-bf10-9553aee30c15-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f23e39e-699c-43ea-bf10-9553aee30c15-workload-socket\") pod \"0f23e39e-699c-43ea-bf10-9553aee30c15\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") "
Oct 30 19:43:22.835102 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.835085    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f23e39e-699c-43ea-bf10-9553aee30c15-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f23e39e-699c-43ea-bf10-9553aee30c15-workload-certs\") pod \"0f23e39e-699c-43ea-bf10-9553aee30c15\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") "
Oct 30 19:43:22.836344 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.835358    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f23e39e-699c-43ea-bf10-9553aee30c15-workload-certs" pod "0f23e39e-699c-43ea-bf10-9553aee30c15" (UID: "0f23e39e-699c-43ea-bf10-9553aee30c15"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:22.836344 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.835398    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0f23e39e-699c-43ea-bf10-9553aee30c15-kube-api-access-ghhbh\" (UniqueName: \"kubernetes.io/projected/0f23e39e-699c-43ea-bf10-9553aee30c15-kube-api-access-ghhbh\") pod \"0f23e39e-699c-43ea-bf10-9553aee30c15\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") "
Oct 30 19:43:22.836344 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.835549    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f23e39e-699c-43ea-bf10-9553aee30c15-workload-socket" pod "0f23e39e-699c-43ea-bf10-9553aee30c15" (UID: "0f23e39e-699c-43ea-bf10-9553aee30c15"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:22.836344 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.835587    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0f23e39e-699c-43ea-bf10-9553aee30c15-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f23e39e-699c-43ea-bf10-9553aee30c15-istiod-ca-cert\") pod \"0f23e39e-699c-43ea-bf10-9553aee30c15\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") "
Oct 30 19:43:22.836344 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.835627    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f23e39e-699c-43ea-bf10-9553aee30c15-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f23e39e-699c-43ea-bf10-9553aee30c15-istio-data\") pod \"0f23e39e-699c-43ea-bf10-9553aee30c15\" (UID: \"0f23e39e-699c-43ea-bf10-9553aee30c15\") "
Oct 30 19:43:22.836344 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.835824    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f23e39e-699c-43ea-bf10-9553aee30c15-credential-socket" pod "0f23e39e-699c-43ea-bf10-9553aee30c15" (UID: "0f23e39e-699c-43ea-bf10-9553aee30c15"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:22.836344 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.836297    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f23e39e-699c-43ea-bf10-9553aee30c15-istio-data" pod "0f23e39e-699c-43ea-bf10-9553aee30c15" (UID: "0f23e39e-699c-43ea-bf10-9553aee30c15"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:22.836825 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.836570    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f23e39e-699c-43ea-bf10-9553aee30c15-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:22.836825 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.836592    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f23e39e-699c-43ea-bf10-9553aee30c15-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:22.836825 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.836607    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f23e39e-699c-43ea-bf10-9553aee30c15-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:22.836825 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.836623    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f23e39e-699c-43ea-bf10-9553aee30c15-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:22.837105 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.836836    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f23e39e-699c-43ea-bf10-9553aee30c15-istiod-ca-cert" pod "0f23e39e-699c-43ea-bf10-9553aee30c15" (UID: "0f23e39e-699c-43ea-bf10-9553aee30c15"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:22.838196 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.838167    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f23e39e-699c-43ea-bf10-9553aee30c15-istio-envoy" pod "0f23e39e-699c-43ea-bf10-9553aee30c15" (UID: "0f23e39e-699c-43ea-bf10-9553aee30c15"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:22.839518 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.839492    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f23e39e-699c-43ea-bf10-9553aee30c15-kube-api-access-ghhbh" pod "0f23e39e-699c-43ea-bf10-9553aee30c15" (UID: "0f23e39e-699c-43ea-bf10-9553aee30c15"). InnerVolumeSpecName "kube-api-access-ghhbh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:22.839778 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.839757    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0f23e39e-699c-43ea-bf10-9553aee30c15-istio-podinfo" pod "0f23e39e-699c-43ea-bf10-9553aee30c15" (UID: "0f23e39e-699c-43ea-bf10-9553aee30c15"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:43:22.840375 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.840351    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f23e39e-699c-43ea-bf10-9553aee30c15-istio-token" pod "0f23e39e-699c-43ea-bf10-9553aee30c15" (UID: "0f23e39e-699c-43ea-bf10-9553aee30c15"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:22.885493 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.885419    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k"
Oct 30 19:43:22.937977 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.937931    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f23e39e-699c-43ea-bf10-9553aee30c15-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:22.938494 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.938138    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f23e39e-699c-43ea-bf10-9553aee30c15-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:22.938494 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.938157    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f23e39e-699c-43ea-bf10-9553aee30c15-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:22.938494 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.938168    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghhbh\" (UniqueName: \"kubernetes.io/projected/0f23e39e-699c-43ea-bf10-9553aee30c15-kube-api-access-ghhbh\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:22.938494 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.938177    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f23e39e-699c-43ea-bf10-9553aee30c15-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:22.952252 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.952219    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"
Oct 30 19:43:22.952468 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.952268    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb" event={"ID":"283578cb-7837-4d4a-b91f-dee1d0d74322","Type":"ContainerDied","Data":"d5b1084784c349828c0b52d257a4d70720882bb38f885031a0ae1f34ef408019"}
Oct 30 19:43:22.952468 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.952332    5907 scope.go:122] "RemoveContainer" containerID="555010fb5775749893e35e327ebf73af2bc1ac9bcd8b0e180889649887577e51"
Oct 30 19:43:22.965891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.954948    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl" event={"ID":"210e8e86-7046-4152-9a79-8f3c0fcdc900","Type":"ContainerDied","Data":"bdec10a710d320d834c5807711c03b7eaafc916e6d6cff5afff4b027651e513d"}
Oct 30 19:43:22.965891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.955092    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"
Oct 30 19:43:22.965891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.960484    5907 generic.go:356] "Generic (PLEG): container finished" podID="0f23e39e-699c-43ea-bf10-9553aee30c15" containerID="06731322ab687584e6abc41e6bf8450d2a702884bb8d69e9266631981f61d373" exitCode=0
Oct 30 19:43:22.965891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.960557    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"
Oct 30 19:43:22.965891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.960573    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j" event={"ID":"0f23e39e-699c-43ea-bf10-9553aee30c15","Type":"ContainerDied","Data":"06731322ab687584e6abc41e6bf8450d2a702884bb8d69e9266631981f61d373"}
Oct 30 19:43:22.965891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.960633    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j" event={"ID":"0f23e39e-699c-43ea-bf10-9553aee30c15","Type":"ContainerDied","Data":"c62c2dfc5f6f451d04dcf692c1a404c0a8df5824f70c58ce2d8bf502265f188f"}
Oct 30 19:43:22.976386 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.969228    5907 scope.go:122] "RemoveContainer" containerID="124ee5b9d32da880da7d58796c17736fda5164ae6c1efc6bb1a6bb003f1a4144"
Oct 30 19:43:22.986965 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.986947    5907 scope.go:122] "RemoveContainer" containerID="06731322ab687584e6abc41e6bf8450d2a702884bb8d69e9266631981f61d373"
Oct 30 19:43:22.998548 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.998525    5907 scope.go:122] "RemoveContainer" containerID="06731322ab687584e6abc41e6bf8450d2a702884bb8d69e9266631981f61d373"
Oct 30 19:43:22.998925 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:43:22.998852    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06731322ab687584e6abc41e6bf8450d2a702884bb8d69e9266631981f61d373\": container with ID starting with 06731322ab687584e6abc41e6bf8450d2a702884bb8d69e9266631981f61d373 not found: ID does not exist" containerID="06731322ab687584e6abc41e6bf8450d2a702884bb8d69e9266631981f61d373"
Oct 30 19:43:22.999003 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:22.998938    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06731322ab687584e6abc41e6bf8450d2a702884bb8d69e9266631981f61d373"} err="failed to get container status \"06731322ab687584e6abc41e6bf8450d2a702884bb8d69e9266631981f61d373\": rpc error: code = NotFound desc = could not find container \"06731322ab687584e6abc41e6bf8450d2a702884bb8d69e9266631981f61d373\": container with ID starting with 06731322ab687584e6abc41e6bf8450d2a702884bb8d69e9266631981f61d373 not found: ID does not exist"
Oct 30 19:43:23.055109 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:23.055076    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"]
Oct 30 19:43:23.063137 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:23.059421    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-xkwnb"]
Oct 30 19:43:23.091638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:23.091583    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"]
Oct 30 19:43:23.095356 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:23.095331    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-7598j"]
Oct 30 19:43:23.139921 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:23.139835    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"]
Oct 30 19:43:23.142155 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:23.142132    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-q2frl"]
Oct 30 19:43:23.587667 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:23.585750    5907 patch_prober.go:29] interesting pod/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.177:15021/healthz/ready\": read tcp 10.134.0.2:44316->10.134.0.177:15021: read: connection reset by peer" start-of-body=
Oct 30 19:43:23.587667 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:23.585805    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd" podUID="dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.177:15021/healthz/ready\": read tcp 10.134.0.2:44316->10.134.0.177:15021: read: connection reset by peer"
Oct 30 19:43:23.972675 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:23.972639    5907 generic.go:356] "Generic (PLEG): container finished" podID="dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe" containerID="45500cbe0d670569b588d9d75bd90ea5d62448e8a0c448d49058575d078886f5" exitCode=0
Oct 30 19:43:23.973147 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:23.972709    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd" event={"ID":"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe","Type":"ContainerDied","Data":"45500cbe0d670569b588d9d75bd90ea5d62448e8a0c448d49058575d078886f5"}
Oct 30 19:43:23.973268 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:23.973248    5907 scope.go:122] "RemoveContainer" containerID="45500cbe0d670569b588d9d75bd90ea5d62448e8a0c448d49058575d078886f5"
Oct 30 19:43:23.977903 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:23.975783    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:43:23.977903 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:23.975982    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:43:23.977903 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:23.976098    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:43:24.290804 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:24.290726    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0f23e39e-699c-43ea-bf10-9553aee30c15" path="/var/lib/kubelet/pods/0f23e39e-699c-43ea-bf10-9553aee30c15/volumes"
Oct 30 19:43:24.291306 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:24.291291    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="210e8e86-7046-4152-9a79-8f3c0fcdc900" path="/var/lib/kubelet/pods/210e8e86-7046-4152-9a79-8f3c0fcdc900/volumes"
Oct 30 19:43:24.291735 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:24.291722    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="283578cb-7837-4d4a-b91f-dee1d0d74322" path="/var/lib/kubelet/pods/283578cb-7837-4d4a-b91f-dee1d0d74322/volumes"
Oct 30 19:43:25.001614 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:25.001574    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd" event={"ID":"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe","Type":"ContainerStarted","Data":"77b47da55d91f085d97dca9028eac1585e92bd7f70c3e1fe58186110e915098d"}
Oct 30 19:43:25.535163 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:25.535123    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:43:25.535726 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:25.535647    5907 patch_prober.go:29] interesting pod/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.177:15021/healthz/ready\": dial tcp 10.134.0.177:15021: connect: connection refused" start-of-body=
Oct 30 19:43:25.535726 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:25.535703    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd" podUID="dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.177:15021/healthz/ready\": dial tcp 10.134.0.177:15021: connect: connection refused"
Oct 30 19:43:26.535299 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:26.535265    5907 patch_prober.go:29] interesting pod/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.177:15021/healthz/ready\": dial tcp 10.134.0.177:15021: connect: connection refused" start-of-body=
Oct 30 19:43:26.535898 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:26.535324    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd" podUID="dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.177:15021/healthz/ready\": dial tcp 10.134.0.177:15021: connect: connection refused"
Oct 30 19:43:27.534888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:27.534649    5907 patch_prober.go:29] interesting pod/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.177:15021/healthz/ready\": dial tcp 10.134.0.177:15021: connect: connection refused" start-of-body=
Oct 30 19:43:27.534888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:27.534709    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd" podUID="dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.177:15021/healthz/ready\": dial tcp 10.134.0.177:15021: connect: connection refused"
Oct 30 19:43:28.354619 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.354582    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"]
Oct 30 19:43:28.355787 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.355764    5907 cpu_manager.go:414] "containerMap: removing container" podUID="210e8e86-7046-4152-9a79-8f3c0fcdc900" containerName="istio-proxy"
Oct 30 19:43:28.355922 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.355791    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="210e8e86-7046-4152-9a79-8f3c0fcdc900" containerName="istio-proxy"
Oct 30 19:43:28.355922 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.355828    5907 cpu_manager.go:414] "containerMap: removing container" podUID="283578cb-7837-4d4a-b91f-dee1d0d74322" containerName="istio-proxy"
Oct 30 19:43:28.355922 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.355835    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="283578cb-7837-4d4a-b91f-dee1d0d74322" containerName="istio-proxy"
Oct 30 19:43:28.355922 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.355878    5907 cpu_manager.go:414] "containerMap: removing container" podUID="0f23e39e-699c-43ea-bf10-9553aee30c15" containerName="istio-proxy"
Oct 30 19:43:28.355922 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.355886    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0f23e39e-699c-43ea-bf10-9553aee30c15" containerName="istio-proxy"
Oct 30 19:43:28.355922 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.355899    5907 cpu_manager.go:414] "containerMap: removing container" podUID="2aa211ac-280d-4294-8f9d-807318fafceb" containerName="revoke-keys"
Oct 30 19:43:28.355922 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.355906    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2aa211ac-280d-4294-8f9d-807318fafceb" containerName="revoke-keys"
Oct 30 19:43:28.356166 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.356133    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="210e8e86-7046-4152-9a79-8f3c0fcdc900" containerName="istio-proxy"
Oct 30 19:43:28.356166 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.356151    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="283578cb-7837-4d4a-b91f-dee1d0d74322" containerName="istio-proxy"
Oct 30 19:43:28.356166 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.356163    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="0f23e39e-699c-43ea-bf10-9553aee30c15" containerName="istio-proxy"
Oct 30 19:43:28.356257 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.356172    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="2aa211ac-280d-4294-8f9d-807318fafceb" containerName="revoke-keys"
Oct 30 19:43:28.362297 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.362276    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"]
Oct 30 19:43:28.362402 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.362383    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:43:28.382785 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.382761    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-3add95-openshift-default-dockercfg-kvxm5\""
Oct 30 19:43:28.622990 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.609269    5907 patch_prober.go:29] interesting pod/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.177:15021/healthz/ready\": dial tcp 10.134.0.177:15021: connect: connection refused" start-of-body=
Oct 30 19:43:28.622990 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.609339    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd" podUID="dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.177:15021/healthz/ready\": dial tcp 10.134.0.177:15021: connect: connection refused"
Oct 30 19:43:28.622990 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.617568    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-workload-certs\") pod \"e2e-api-a-3add95-openshift-default-5d656856f-rt2nh\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") " pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:43:28.622990 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.617654    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-istiod-ca-cert\") pod \"e2e-api-a-3add95-openshift-default-5d656856f-rt2nh\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") " pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:43:28.622990 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.617853    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-istio-token\") pod \"e2e-api-a-3add95-openshift-default-5d656856f-rt2nh\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") " pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:43:28.622990 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.618173    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-istio-podinfo\") pod \"e2e-api-a-3add95-openshift-default-5d656856f-rt2nh\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") " pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:43:28.664300 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.664259    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-istio-data\") pod \"e2e-api-a-3add95-openshift-default-5d656856f-rt2nh\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") " pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:43:28.669953 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.664611    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-istio-envoy\") pod \"e2e-api-a-3add95-openshift-default-5d656856f-rt2nh\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") " pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:43:28.669953 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.664684    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddkrs\" (UniqueName: \"kubernetes.io/projected/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-kube-api-access-ddkrs\") pod \"e2e-api-a-3add95-openshift-default-5d656856f-rt2nh\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") " pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:43:28.669953 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.664782    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-workload-socket\") pod \"e2e-api-a-3add95-openshift-default-5d656856f-rt2nh\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") " pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:43:28.669953 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.664894    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-credential-socket\") pod \"e2e-api-a-3add95-openshift-default-5d656856f-rt2nh\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") " pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:43:28.808748 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.808708    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-istio-podinfo\") pod \"e2e-api-a-3add95-openshift-default-5d656856f-rt2nh\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") " pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:43:28.808965 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.808807    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-istio-data\") pod \"e2e-api-a-3add95-openshift-default-5d656856f-rt2nh\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") " pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:43:28.808965 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.808894    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-istio-envoy\") pod \"e2e-api-a-3add95-openshift-default-5d656856f-rt2nh\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") " pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:43:28.808965 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.808958    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddkrs\" (UniqueName: \"kubernetes.io/projected/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-kube-api-access-ddkrs\") pod \"e2e-api-a-3add95-openshift-default-5d656856f-rt2nh\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") " pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:43:28.809172 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.809056    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-workload-socket\") pod \"e2e-api-a-3add95-openshift-default-5d656856f-rt2nh\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") " pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:43:28.809172 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.809134    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-credential-socket\") pod \"e2e-api-a-3add95-openshift-default-5d656856f-rt2nh\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") " pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:43:28.809289 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.809227    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-workload-certs\") pod \"e2e-api-a-3add95-openshift-default-5d656856f-rt2nh\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") " pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:43:28.809342 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.809285    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-istiod-ca-cert\") pod \"e2e-api-a-3add95-openshift-default-5d656856f-rt2nh\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") " pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:43:28.809401 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.809339    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-istio-token\") pod \"e2e-api-a-3add95-openshift-default-5d656856f-rt2nh\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") " pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:43:28.809805 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.809775    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-credential-socket\") pod \"e2e-api-a-3add95-openshift-default-5d656856f-rt2nh\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") " pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:43:28.809965 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.809850    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-istio-data\") pod \"e2e-api-a-3add95-openshift-default-5d656856f-rt2nh\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") " pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:43:28.823447 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.810103    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-workload-socket\") pod \"e2e-api-a-3add95-openshift-default-5d656856f-rt2nh\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") " pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:43:28.823447 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.810373    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-workload-certs\") pod \"e2e-api-a-3add95-openshift-default-5d656856f-rt2nh\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") " pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:43:28.823447 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.812741    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-istiod-ca-cert\") pod \"e2e-api-a-3add95-openshift-default-5d656856f-rt2nh\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") " pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:43:28.842753 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.842713    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-istio-token\") pod \"e2e-api-a-3add95-openshift-default-5d656856f-rt2nh\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") " pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:43:28.843561 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.843538    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-istio-podinfo\") pod \"e2e-api-a-3add95-openshift-default-5d656856f-rt2nh\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") " pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:43:28.865791 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.847567    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-istio-envoy\") pod \"e2e-api-a-3add95-openshift-default-5d656856f-rt2nh\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") " pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:43:28.865791 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:28.850488    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddkrs\" (UniqueName: \"kubernetes.io/projected/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-kube-api-access-ddkrs\") pod \"e2e-api-a-3add95-openshift-default-5d656856f-rt2nh\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") " pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:43:29.103151 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:29.103061    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:43:29.462835 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:29.462802    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"]
Oct 30 19:43:29.464520 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:43:29.464499    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5678ec4_1e40_4b66_a9d2_7e6bf08db0bf.slice/crio-368654e63ab0482f8098e8620b1130eb9d6bda7e6105070fe74607fb77c14a8c WatchSource:0}: Error finding container 368654e63ab0482f8098e8620b1130eb9d6bda7e6105070fe74607fb77c14a8c: Status 404 returned error can't find the container with id 368654e63ab0482f8098e8620b1130eb9d6bda7e6105070fe74607fb77c14a8c
Oct 30 19:43:29.467951 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:29.467915    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:43:29.468048 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:29.467987    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:43:29.468048 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:29.468015    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:43:29.535358 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:29.535326    5907 patch_prober.go:29] interesting pod/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.177:15021/healthz/ready\": dial tcp 10.134.0.177:15021: connect: connection refused" start-of-body=
Oct 30 19:43:29.535567 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:29.535538    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd" podUID="dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.177:15021/healthz/ready\": dial tcp 10.134.0.177:15021: connect: connection refused"
Oct 30 19:43:30.072096 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:30.072055    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh" event={"ID":"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf","Type":"ContainerStarted","Data":"28b30d77ae7ed333981688e96cd6f906eea4c288ff68b8efb0764bbfe4fd7b65"}
Oct 30 19:43:30.072290 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:30.072119    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh" event={"ID":"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf","Type":"ContainerStarted","Data":"368654e63ab0482f8098e8620b1130eb9d6bda7e6105070fe74607fb77c14a8c"}
Oct 30 19:43:30.112885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:30.108507    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:43:30.126964 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:30.125236    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:43:30.190632 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:30.184819    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh" podStartSLOduration=2.184794651 podStartE2EDuration="2.184794651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:43:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:43:30.148366682 +0000 UTC m=+1321.991298627" watchObservedRunningTime="2026-10-30 19:43:30.184794651 +0000 UTC m=+1322.027726591"
Oct 30 19:43:30.266961 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:30.261628    5907 scope.go:122] "RemoveContainer" containerID="a8a36f569ab54a83a98e5e6a42263cd1534106d3f67e380d04e0e00db366984d"
Oct 30 19:43:30.331951 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:30.328028    5907 scope.go:122] "RemoveContainer" containerID="123f3de9c7f069ff2b3f320ea8e6499f343b0d355ae8df6f5a86d880325cab24"
Oct 30 19:43:30.438799 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:30.438753    5907 scope.go:122] "RemoveContainer" containerID="c2fddce35e7897a9d39d52e98ae474922b7b9c88909421f046298d67ea5c4f5b"
Oct 30 19:43:30.456744 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:30.456720    5907 scope.go:122] "RemoveContainer" containerID="c9ac9cff65596d02f3dbe7eddaa21b1225ccccd830c8eb4ff8ce678012a57032"
Oct 30 19:43:30.469623 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:30.469528    5907 scope.go:122] "RemoveContainer" containerID="605e663001b21092383ab253c830b8c26839fb9e1edfdd7a651117546d9c02e1"
Oct 30 19:43:30.483587 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:30.483563    5907 scope.go:122] "RemoveContainer" containerID="8773732a20df86c8808c02fc130493ca9287771283707c1351381cb6aaeab47e"
Oct 30 19:43:30.496819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:30.496789    5907 scope.go:122] "RemoveContainer" containerID="175b0ac83636e672c808236b06efa38eb3d1bcee8060a8cec2a62ac3dbbba9be"
Oct 30 19:43:30.507598 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:30.507577    5907 scope.go:122] "RemoveContainer" containerID="7535bc14346ebcf460fc6daee2c4df11b713378cd598b05e2e45a268f9a1700d"
Oct 30 19:43:30.518414 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:30.518395    5907 scope.go:122] "RemoveContainer" containerID="76aa4885a65c7ed3acaed85656f95ac1402fd00135cb8b87caadbd5940167003"
Oct 30 19:43:30.529029 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:30.528985    5907 scope.go:122] "RemoveContainer" containerID="198c1d15feb8796cedd11964f77482afd070de6edd30b95afe6c66e6e5ab3996"
Oct 30 19:43:30.535438 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:30.535415    5907 patch_prober.go:29] interesting pod/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.177:15021/healthz/ready\": dial tcp 10.134.0.177:15021: connect: connection refused" start-of-body=
Oct 30 19:43:30.535568 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:30.535466    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd" podUID="dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.177:15021/healthz/ready\": dial tcp 10.134.0.177:15021: connect: connection refused"
Oct 30 19:43:30.539751 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:30.539727    5907 scope.go:122] "RemoveContainer" containerID="454adf472db5f76eb43e65fa2f344d81bd761a4fdf2696fe788be676351256c0"
Oct 30 19:43:30.551400 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:30.551376    5907 scope.go:122] "RemoveContainer" containerID="66ed25c2651916d246bac929ff924f7fa288a9b88a47da69c885cc6e058717d1"
Oct 30 19:43:30.563424 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:30.563397    5907 scope.go:122] "RemoveContainer" containerID="312a31e39c0f98a2aaef81c13587743a31013e69ccc3080e9a6e3a00f5b41edc"
Oct 30 19:43:31.080360 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:31.080327    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:43:31.081244 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:31.081224    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:43:31.540439 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:31.540393    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:43:31.540967 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:31.540915    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:43:31.541789 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:31.541768    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:43:33.662840 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:33.662797    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq"]
Oct 30 19:43:33.668960 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:33.668938    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq"
Oct 30 19:43:33.696133 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:33.693439    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq"]
Oct 30 19:43:33.696133 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:33.693625    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w0-0c305f-serving-cert\""
Oct 30 19:43:33.748849 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:33.748819    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55kjq\" (UniqueName: \"kubernetes.io/projected/d71d8254-621d-4cac-94ea-28c912d5a45b-kube-api-access-55kjq\") pod \"maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq\" (UID: \"d71d8254-621d-4cac-94ea-28c912d5a45b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq"
Oct 30 19:43:33.749027 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:33.748991    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d71d8254-621d-4cac-94ea-28c912d5a45b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq\" (UID: \"d71d8254-621d-4cac-94ea-28c912d5a45b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq"
Oct 30 19:43:33.749066 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:33.749035    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d71d8254-621d-4cac-94ea-28c912d5a45b-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq\" (UID: \"d71d8254-621d-4cac-94ea-28c912d5a45b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq"
Oct 30 19:43:33.749265 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:33.749236    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d71d8254-621d-4cac-94ea-28c912d5a45b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq\" (UID: \"d71d8254-621d-4cac-94ea-28c912d5a45b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq"
Oct 30 19:43:33.850655 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:33.850620    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d71d8254-621d-4cac-94ea-28c912d5a45b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq\" (UID: \"d71d8254-621d-4cac-94ea-28c912d5a45b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq"
Oct 30 19:43:33.850908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:33.850808    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55kjq\" (UniqueName: \"kubernetes.io/projected/d71d8254-621d-4cac-94ea-28c912d5a45b-kube-api-access-55kjq\") pod \"maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq\" (UID: \"d71d8254-621d-4cac-94ea-28c912d5a45b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq"
Oct 30 19:43:33.851015 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:33.850925    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d71d8254-621d-4cac-94ea-28c912d5a45b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq\" (UID: \"d71d8254-621d-4cac-94ea-28c912d5a45b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq"
Oct 30 19:43:33.851015 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:33.850959    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d71d8254-621d-4cac-94ea-28c912d5a45b-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq\" (UID: \"d71d8254-621d-4cac-94ea-28c912d5a45b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq"
Oct 30 19:43:33.851391 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:33.851364    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d71d8254-621d-4cac-94ea-28c912d5a45b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq\" (UID: \"d71d8254-621d-4cac-94ea-28c912d5a45b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq"
Oct 30 19:43:33.853798 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:33.853774    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d71d8254-621d-4cac-94ea-28c912d5a45b-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq\" (UID: \"d71d8254-621d-4cac-94ea-28c912d5a45b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq"
Oct 30 19:43:33.853915 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:33.853855    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d71d8254-621d-4cac-94ea-28c912d5a45b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq\" (UID: \"d71d8254-621d-4cac-94ea-28c912d5a45b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq"
Oct 30 19:43:33.925019 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:33.923455    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55kjq\" (UniqueName: \"kubernetes.io/projected/d71d8254-621d-4cac-94ea-28c912d5a45b-kube-api-access-55kjq\") pod \"maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq\" (UID: \"d71d8254-621d-4cac-94ea-28c912d5a45b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq"
Oct 30 19:43:34.001886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:33.987267    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq"
Oct 30 19:43:34.360125 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:34.358803    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq"]
Oct 30 19:43:34.759383 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:34.759349    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-3add95-5f665b5b94-hdzl8"]
Oct 30 19:43:34.763407 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:34.763386    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-3add95-5f665b5b94-hdzl8"
Oct 30 19:43:34.782498 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:34.782477    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-3add95-dockercfg-dp2cf\""
Oct 30 19:43:34.782600 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:34.782485    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-3add95\""
Oct 30 19:43:34.790735 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:34.790707    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-3add95-5f665b5b94-hdzl8"]
Oct 30 19:43:34.824474 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:34.824449    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-3add95-d9bb45d9f-sjmpj"]
Oct 30 19:43:34.829691 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:34.829664    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3add95-d9bb45d9f-sjmpj"
Oct 30 19:43:34.852508 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:34.851191    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-3add95-d9bb45d9f-sjmpj"]
Oct 30 19:43:34.994883 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:34.994419    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h46mx\" (UniqueName: \"kubernetes.io/projected/7cd1ce0b-3b85-4701-9f18-abdf8a83e564-kube-api-access-h46mx\") pod \"payload-processing-e2e-api-a-3add95-5f665b5b94-hdzl8\" (UID: \"7cd1ce0b-3b85-4701-9f18-abdf8a83e564\") " pod="openshift-ingress/payload-processing-e2e-api-a-3add95-5f665b5b94-hdzl8"
Oct 30 19:43:34.994883 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:34.994534    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7cd1ce0b-3b85-4701-9f18-abdf8a83e564-plugins-config-volume\") pod \"payload-processing-e2e-api-a-3add95-5f665b5b94-hdzl8\" (UID: \"7cd1ce0b-3b85-4701-9f18-abdf8a83e564\") " pod="openshift-ingress/payload-processing-e2e-api-a-3add95-5f665b5b94-hdzl8"
Oct 30 19:43:35.096945 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:35.096795    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h46mx\" (UniqueName: \"kubernetes.io/projected/7cd1ce0b-3b85-4701-9f18-abdf8a83e564-kube-api-access-h46mx\") pod \"payload-processing-e2e-api-a-3add95-5f665b5b94-hdzl8\" (UID: \"7cd1ce0b-3b85-4701-9f18-abdf8a83e564\") " pod="openshift-ingress/payload-processing-e2e-api-a-3add95-5f665b5b94-hdzl8"
Oct 30 19:43:35.097255 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:35.097232    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7cd1ce0b-3b85-4701-9f18-abdf8a83e564-plugins-config-volume\") pod \"payload-processing-e2e-api-a-3add95-5f665b5b94-hdzl8\" (UID: \"7cd1ce0b-3b85-4701-9f18-abdf8a83e564\") " pod="openshift-ingress/payload-processing-e2e-api-a-3add95-5f665b5b94-hdzl8"
Oct 30 19:43:35.097381 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:35.097365    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4e830ac-bc83-4389-ac35-f00e5fbdf258-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-3add95-d9bb45d9f-sjmpj\" (UID: \"c4e830ac-bc83-4389-ac35-f00e5fbdf258\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-3add95-d9bb45d9f-sjmpj"
Oct 30 19:43:35.097853 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:35.097832    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d96rq\" (UniqueName: \"kubernetes.io/projected/c4e830ac-bc83-4389-ac35-f00e5fbdf258-kube-api-access-d96rq\") pod \"payload-pre-processing-e2e-api-a-3add95-d9bb45d9f-sjmpj\" (UID: \"c4e830ac-bc83-4389-ac35-f00e5fbdf258\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-3add95-d9bb45d9f-sjmpj"
Oct 30 19:43:35.099163 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:35.099138    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7cd1ce0b-3b85-4701-9f18-abdf8a83e564-plugins-config-volume\") pod \"payload-processing-e2e-api-a-3add95-5f665b5b94-hdzl8\" (UID: \"7cd1ce0b-3b85-4701-9f18-abdf8a83e564\") " pod="openshift-ingress/payload-processing-e2e-api-a-3add95-5f665b5b94-hdzl8"
Oct 30 19:43:35.157814 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:35.155561    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h46mx\" (UniqueName: \"kubernetes.io/projected/7cd1ce0b-3b85-4701-9f18-abdf8a83e564-kube-api-access-h46mx\") pod \"payload-processing-e2e-api-a-3add95-5f665b5b94-hdzl8\" (UID: \"7cd1ce0b-3b85-4701-9f18-abdf8a83e564\") " pod="openshift-ingress/payload-processing-e2e-api-a-3add95-5f665b5b94-hdzl8"
Oct 30 19:43:35.172526 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:35.170317    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq" event={"ID":"d71d8254-621d-4cac-94ea-28c912d5a45b","Type":"ContainerStarted","Data":"20239eca8c288234216eab29e978f7e986ebef29c49cde68ed39bab6e49ef441"}
Oct 30 19:43:35.172526 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:35.170390    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq" event={"ID":"d71d8254-621d-4cac-94ea-28c912d5a45b","Type":"ContainerStarted","Data":"5b04a77650c9b23832ce02f4d0c68d5eb765cd63c05b43f2f2e81528617c9552"}
Oct 30 19:43:35.174144 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:35.174059    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq"
Oct 30 19:43:35.201914 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:35.200038    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d96rq\" (UniqueName: \"kubernetes.io/projected/c4e830ac-bc83-4389-ac35-f00e5fbdf258-kube-api-access-d96rq\") pod \"payload-pre-processing-e2e-api-a-3add95-d9bb45d9f-sjmpj\" (UID: \"c4e830ac-bc83-4389-ac35-f00e5fbdf258\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-3add95-d9bb45d9f-sjmpj"
Oct 30 19:43:35.201914 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:35.200245    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4e830ac-bc83-4389-ac35-f00e5fbdf258-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-3add95-d9bb45d9f-sjmpj\" (UID: \"c4e830ac-bc83-4389-ac35-f00e5fbdf258\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-3add95-d9bb45d9f-sjmpj"
Oct 30 19:43:35.219803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:35.219760    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4e830ac-bc83-4389-ac35-f00e5fbdf258-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-3add95-d9bb45d9f-sjmpj\" (UID: \"c4e830ac-bc83-4389-ac35-f00e5fbdf258\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-3add95-d9bb45d9f-sjmpj"
Oct 30 19:43:35.245175 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:35.245110    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq" podStartSLOduration=1.837126451 podStartE2EDuration="2.245088437s" totalImagesPullingTime="407.961986ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:43:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:43:35.228643623 +0000 UTC m=+1327.071575560" watchObservedRunningTime="2026-10-30 19:43:35.245088437 +0000 UTC m=+1327.088020373"
Oct 30 19:43:35.251739 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:35.251705    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d96rq\" (UniqueName: \"kubernetes.io/projected/c4e830ac-bc83-4389-ac35-f00e5fbdf258-kube-api-access-d96rq\") pod \"payload-pre-processing-e2e-api-a-3add95-d9bb45d9f-sjmpj\" (UID: \"c4e830ac-bc83-4389-ac35-f00e5fbdf258\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-3add95-d9bb45d9f-sjmpj"
Oct 30 19:43:35.378848 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:35.378814    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-3add95-5f665b5b94-hdzl8"
Oct 30 19:43:35.448438 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:35.448411    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3add95-d9bb45d9f-sjmpj"
Oct 30 19:43:35.834006 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:35.830650    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-3add95-5f665b5b94-hdzl8"]
Oct 30 19:43:36.156703 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:36.156640    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7e89da-t5hgg"]
Oct 30 19:43:36.157009 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:36.156992    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7e89da-t5hgg"]
Oct 30 19:43:36.258417 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:36.255071    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-3add95-d9bb45d9f-sjmpj"]
Oct 30 19:43:36.268031 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:36.267988    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-3add95-5f665b5b94-hdzl8" event={"ID":"7cd1ce0b-3b85-4701-9f18-abdf8a83e564","Type":"ContainerStarted","Data":"0a6c56840cc8c322366b8bfedb46dda7ae549c87934781c37df6ed5ff4fb498b"}
Oct 30 19:43:36.349412 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:36.349357    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="713310f5-4d82-4e18-877d-235cfac28327" path="/var/lib/kubelet/pods/713310f5-4d82-4e18-877d-235cfac28327/volumes"
Oct 30 19:43:37.276745 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:37.276704    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3add95-d9bb45d9f-sjmpj" event={"ID":"c4e830ac-bc83-4389-ac35-f00e5fbdf258","Type":"ContainerStarted","Data":"886f52afe95b72986f54c7d26f121ad65c2a4779643e1b364f0f987a93972222"}
Oct 30 19:43:37.277231 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:37.276796    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3add95-d9bb45d9f-sjmpj"
Oct 30 19:43:37.277231 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:37.276820    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3add95-d9bb45d9f-sjmpj" event={"ID":"c4e830ac-bc83-4389-ac35-f00e5fbdf258","Type":"ContainerStarted","Data":"31de82f327bad0278c410c6d7be5a3d0ee775d67474029931a3610037a3e9380"}
Oct 30 19:43:37.278545 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:37.278518    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-3add95-5f665b5b94-hdzl8" event={"ID":"7cd1ce0b-3b85-4701-9f18-abdf8a83e564","Type":"ContainerStarted","Data":"d5b866d9b799e30cde92d90bbacd5c90e0524c681da7f081195e7cbd5c7f85c7"}
Oct 30 19:43:37.278691 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:37.278604    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-3add95-5f665b5b94-hdzl8"
Oct 30 19:43:37.341569 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:37.341506    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3add95-d9bb45d9f-sjmpj" podStartSLOduration=3.3414923180000002 podStartE2EDuration="3.341492318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:43:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:43:37.336533693 +0000 UTC m=+1329.179465632" watchObservedRunningTime="2026-10-30 19:43:37.341492318 +0000 UTC m=+1329.184424251"
Oct 30 19:43:37.389405 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:37.389357    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-3add95-5f665b5b94-hdzl8" podStartSLOduration=3.389343078 podStartE2EDuration="3.389343078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:43:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:43:37.387584618 +0000 UTC m=+1329.230516558" watchObservedRunningTime="2026-10-30 19:43:37.389343078 +0000 UTC m=+1329.232275017"
Oct 30 19:43:42.292564 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:42.292530    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq"
Oct 30 19:43:43.856424 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:43.856367    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"]
Oct 30 19:43:43.857569 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:43.857504    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp" podUID="29fbb7be-9068-40ec-a0d4-9fa0ed0469bf" containerName="istio-proxy" containerID="cri-o://7d3b1f55131c855d23bf3ff42421397abc230221ce98296c182f1e26ace045d6" gracePeriod=30
Oct 30 19:43:44.696342 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:44.696298    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-dd23d1-mngxw"]
Oct 30 19:43:44.701251 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:44.701230    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-dd23d1-mngxw"
Oct 30 19:43:44.724804 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:44.724773    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:43:44.731234 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:44.731210    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:43:44.735655 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:44.735627    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-dd23d1-mngxw"]
Oct 30 19:43:44.830994 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:44.830963    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9e9a9805-721e-4939-a8c4-d078d6a3dc32-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-dd23d1-mngxw\" (UID: \"9e9a9805-721e-4939-a8c4-d078d6a3dc32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-dd23d1-mngxw"
Oct 30 19:43:44.831204 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:44.831023    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9e9a9805-721e-4939-a8c4-d078d6a3dc32-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-dd23d1-mngxw\" (UID: \"9e9a9805-721e-4939-a8c4-d078d6a3dc32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-dd23d1-mngxw"
Oct 30 19:43:44.831204 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:44.831113    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh8ch\" (UniqueName: \"kubernetes.io/projected/9e9a9805-721e-4939-a8c4-d078d6a3dc32-kube-api-access-zh8ch\") pod \"maas-api-revoke-keys-e2e-worker-w3-dd23d1-mngxw\" (UID: \"9e9a9805-721e-4939-a8c4-d078d6a3dc32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-dd23d1-mngxw"
Oct 30 19:43:44.932728 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:44.932687    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9e9a9805-721e-4939-a8c4-d078d6a3dc32-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-dd23d1-mngxw\" (UID: \"9e9a9805-721e-4939-a8c4-d078d6a3dc32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-dd23d1-mngxw"
Oct 30 19:43:44.933243 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:44.932767    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9e9a9805-721e-4939-a8c4-d078d6a3dc32-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-dd23d1-mngxw\" (UID: \"9e9a9805-721e-4939-a8c4-d078d6a3dc32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-dd23d1-mngxw"
Oct 30 19:43:44.933243 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:44.932791    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh8ch\" (UniqueName: \"kubernetes.io/projected/9e9a9805-721e-4939-a8c4-d078d6a3dc32-kube-api-access-zh8ch\") pod \"maas-api-revoke-keys-e2e-worker-w3-dd23d1-mngxw\" (UID: \"9e9a9805-721e-4939-a8c4-d078d6a3dc32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-dd23d1-mngxw"
Oct 30 19:43:44.933470 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:44.933449    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9e9a9805-721e-4939-a8c4-d078d6a3dc32-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-dd23d1-mngxw\" (UID: \"9e9a9805-721e-4939-a8c4-d078d6a3dc32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-dd23d1-mngxw"
Oct 30 19:43:44.935391 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:44.935368    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9e9a9805-721e-4939-a8c4-d078d6a3dc32-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-dd23d1-mngxw\" (UID: \"9e9a9805-721e-4939-a8c4-d078d6a3dc32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-dd23d1-mngxw"
Oct 30 19:43:44.954638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:44.954577    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh8ch\" (UniqueName: \"kubernetes.io/projected/9e9a9805-721e-4939-a8c4-d078d6a3dc32-kube-api-access-zh8ch\") pod \"maas-api-revoke-keys-e2e-worker-w3-dd23d1-mngxw\" (UID: \"9e9a9805-721e-4939-a8c4-d078d6a3dc32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-dd23d1-mngxw"
Oct 30 19:43:45.015979 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:45.015950    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-dd23d1-mngxw"
Oct 30 19:43:45.353720 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:43:45.353686    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e9a9805_721e_4939_a8c4_d078d6a3dc32.slice/crio-5ecc0eb6d0bcbaf872c3a2674ea16127551c74645b0837236c3609e3742f4ab3 WatchSource:0}: Error finding container 5ecc0eb6d0bcbaf872c3a2674ea16127551c74645b0837236c3609e3742f4ab3: Status 404 returned error can't find the container with id 5ecc0eb6d0bcbaf872c3a2674ea16127551c74645b0837236c3609e3742f4ab3
Oct 30 19:43:45.369969 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:45.369932    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-dd23d1-mngxw"]
Oct 30 19:43:45.455576 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:45.455546    5907 patch_prober.go:29] interesting pod/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:43:45.455713 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:45.455602    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp" podUID="29fbb7be-9068-40ec-a0d4-9fa0ed0469bf" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:43:46.138940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:46.138909    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4a4e2f-vp8vp"]
Oct 30 19:43:46.143668 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:46.143646    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4a4e2f-vp8vp"
Oct 30 19:43:46.166030 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:46.165999    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4a4e2f-vp8vp"]
Oct 30 19:43:46.180328 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:46.180291    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/80b91a64-9588-4a51-a88e-c0fc49052ab1-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-api-a-4a4e2f-vp8vp\" (UID: \"80b91a64-9588-4a51-a88e-c0fc49052ab1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4a4e2f-vp8vp"
Oct 30 19:43:46.180623 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:46.180597    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/80b91a64-9588-4a51-a88e-c0fc49052ab1-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-api-a-4a4e2f-vp8vp\" (UID: \"80b91a64-9588-4a51-a88e-c0fc49052ab1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4a4e2f-vp8vp"
Oct 30 19:43:46.180756 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:46.180717    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bmxt\" (UniqueName: \"kubernetes.io/projected/80b91a64-9588-4a51-a88e-c0fc49052ab1-kube-api-access-6bmxt\") pod \"maas-api-revoke-keys-e2e-api-a-4a4e2f-vp8vp\" (UID: \"80b91a64-9588-4a51-a88e-c0fc49052ab1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4a4e2f-vp8vp"
Oct 30 19:43:46.281897 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:46.281847    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/80b91a64-9588-4a51-a88e-c0fc49052ab1-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-api-a-4a4e2f-vp8vp\" (UID: \"80b91a64-9588-4a51-a88e-c0fc49052ab1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4a4e2f-vp8vp"
Oct 30 19:43:46.282061 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:46.281947    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bmxt\" (UniqueName: \"kubernetes.io/projected/80b91a64-9588-4a51-a88e-c0fc49052ab1-kube-api-access-6bmxt\") pod \"maas-api-revoke-keys-e2e-api-a-4a4e2f-vp8vp\" (UID: \"80b91a64-9588-4a51-a88e-c0fc49052ab1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4a4e2f-vp8vp"
Oct 30 19:43:46.282125 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:46.282076    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/80b91a64-9588-4a51-a88e-c0fc49052ab1-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-api-a-4a4e2f-vp8vp\" (UID: \"80b91a64-9588-4a51-a88e-c0fc49052ab1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4a4e2f-vp8vp"
Oct 30 19:43:46.282585 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:46.282557    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/80b91a64-9588-4a51-a88e-c0fc49052ab1-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-api-a-4a4e2f-vp8vp\" (UID: \"80b91a64-9588-4a51-a88e-c0fc49052ab1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4a4e2f-vp8vp"
Oct 30 19:43:46.284669 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:46.284651    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/80b91a64-9588-4a51-a88e-c0fc49052ab1-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-api-a-4a4e2f-vp8vp\" (UID: \"80b91a64-9588-4a51-a88e-c0fc49052ab1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4a4e2f-vp8vp"
Oct 30 19:43:46.308619 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:46.308591    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bmxt\" (UniqueName: \"kubernetes.io/projected/80b91a64-9588-4a51-a88e-c0fc49052ab1-kube-api-access-6bmxt\") pod \"maas-api-revoke-keys-e2e-api-a-4a4e2f-vp8vp\" (UID: \"80b91a64-9588-4a51-a88e-c0fc49052ab1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4a4e2f-vp8vp"
Oct 30 19:43:46.358959 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:46.358919    5907 generic.go:356] "Generic (PLEG): container finished" podID="9e9a9805-721e-4939-a8c4-d078d6a3dc32" containerID="920ad0de7585d8fb7f5ace99626274b09aeb1ce927a09a0285850214e7eb1740" exitCode=0
Oct 30 19:43:46.359118 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:46.359011    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-dd23d1-mngxw" event={"ID":"9e9a9805-721e-4939-a8c4-d078d6a3dc32","Type":"ContainerDied","Data":"920ad0de7585d8fb7f5ace99626274b09aeb1ce927a09a0285850214e7eb1740"}
Oct 30 19:43:46.359118 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:46.359069    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-dd23d1-mngxw" event={"ID":"9e9a9805-721e-4939-a8c4-d078d6a3dc32","Type":"ContainerStarted","Data":"5ecc0eb6d0bcbaf872c3a2674ea16127551c74645b0837236c3609e3742f4ab3"}
Oct 30 19:43:46.460019 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:46.459926    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4a4e2f-vp8vp"
Oct 30 19:43:46.752524 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:46.752453    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"]
Oct 30 19:43:46.752677 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:46.752649    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv" podUID="01992c2f-0e71-4b86-8e8a-69bb7727bd6a" containerName="istio-proxy" containerID="cri-o://517f1f95ec7893dc438e29d7c93505719922128a86c72f062d50868d00705195" gracePeriod=30
Oct 30 19:43:46.766065 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:46.766036    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"]
Oct 30 19:43:46.766295 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:46.766269    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq" podUID="b6947d3d-3c9a-40d6-a9ae-0604ec1c7125" containerName="istio-proxy" containerID="cri-o://e7c613909fa4ea2bd315211f18eb37a63655bee217949710516c09dd1ff9a3fa" gracePeriod=30
Oct 30 19:43:46.771174 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:46.771139    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"]
Oct 30 19:43:46.771412 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:46.771336    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4" podUID="27e1950d-b453-4f32-afd2-95ab9027a2e5" containerName="istio-proxy" containerID="cri-o://6ef915f651c8f45e0b04b63b9bb1e6c7d25192c102ac8f92036f1cea6a57e038" gracePeriod=30
Oct 30 19:43:46.774770 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:46.774746    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"]
Oct 30 19:43:46.777290 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:46.774951    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np" podUID="71285665-ae23-4783-a24f-535dccfb68c7" containerName="istio-proxy" containerID="cri-o://7e0b09bd4ae355d8392bff6d6599fdfe47e0360a686924bf9de3595f210fff14" gracePeriod=30
Oct 30 19:43:46.783194 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:46.783169    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4a4e2f-vp8vp"]
Oct 30 19:43:46.783399 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:43:46.783378    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80b91a64_9588_4a51_a88e_c0fc49052ab1.slice/crio-e38ea11bd2ece1b6c09d0db0d9fe8b5d960c5afb63a9b881a4ec1286255be7b6 WatchSource:0}: Error finding container e38ea11bd2ece1b6c09d0db0d9fe8b5d960c5afb63a9b881a4ec1286255be7b6: Status 404 returned error can't find the container with id e38ea11bd2ece1b6c09d0db0d9fe8b5d960c5afb63a9b881a4ec1286255be7b6
Oct 30 19:43:46.991685 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:46.991638    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-28885407-b2tx8"]
Oct 30 19:43:47.073281 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:47.069539    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-28885407-b2tx8"]
Oct 30 19:43:47.073281 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:47.069682    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-28885407-b2tx8"
Oct 30 19:43:47.096612 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:47.096578    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/eb6373d5-f4d9-4110-b9f9-2e1968021aee-maas-api-xks-ca\") pod \"maas-api-revoke-sub-28885407-b2tx8\" (UID: \"eb6373d5-f4d9-4110-b9f9-2e1968021aee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-28885407-b2tx8"
Oct 30 19:43:47.096750 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:47.096723    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/eb6373d5-f4d9-4110-b9f9-2e1968021aee-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-28885407-b2tx8\" (UID: \"eb6373d5-f4d9-4110-b9f9-2e1968021aee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-28885407-b2tx8"
Oct 30 19:43:47.096837 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:47.096783    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5b6b4\" (UniqueName: \"kubernetes.io/projected/eb6373d5-f4d9-4110-b9f9-2e1968021aee-kube-api-access-5b6b4\") pod \"maas-api-revoke-sub-28885407-b2tx8\" (UID: \"eb6373d5-f4d9-4110-b9f9-2e1968021aee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-28885407-b2tx8"
Oct 30 19:43:47.198362 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:47.198333    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/eb6373d5-f4d9-4110-b9f9-2e1968021aee-maas-api-xks-ca\") pod \"maas-api-revoke-sub-28885407-b2tx8\" (UID: \"eb6373d5-f4d9-4110-b9f9-2e1968021aee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-28885407-b2tx8"
Oct 30 19:43:47.198697 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:47.198393    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/eb6373d5-f4d9-4110-b9f9-2e1968021aee-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-28885407-b2tx8\" (UID: \"eb6373d5-f4d9-4110-b9f9-2e1968021aee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-28885407-b2tx8"
Oct 30 19:43:47.198697 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:47.198436    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5b6b4\" (UniqueName: \"kubernetes.io/projected/eb6373d5-f4d9-4110-b9f9-2e1968021aee-kube-api-access-5b6b4\") pod \"maas-api-revoke-sub-28885407-b2tx8\" (UID: \"eb6373d5-f4d9-4110-b9f9-2e1968021aee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-28885407-b2tx8"
Oct 30 19:43:47.199150 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:47.199083    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/eb6373d5-f4d9-4110-b9f9-2e1968021aee-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-28885407-b2tx8\" (UID: \"eb6373d5-f4d9-4110-b9f9-2e1968021aee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-28885407-b2tx8"
Oct 30 19:43:47.200839 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:47.200815    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/eb6373d5-f4d9-4110-b9f9-2e1968021aee-maas-api-xks-ca\") pod \"maas-api-revoke-sub-28885407-b2tx8\" (UID: \"eb6373d5-f4d9-4110-b9f9-2e1968021aee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-28885407-b2tx8"
Oct 30 19:43:47.248299 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:47.248275    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5b6b4\" (UniqueName: \"kubernetes.io/projected/eb6373d5-f4d9-4110-b9f9-2e1968021aee-kube-api-access-5b6b4\") pod \"maas-api-revoke-sub-28885407-b2tx8\" (UID: \"eb6373d5-f4d9-4110-b9f9-2e1968021aee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-28885407-b2tx8"
Oct 30 19:43:47.377712 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:47.372806    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4a4e2f-vp8vp" event={"ID":"80b91a64-9588-4a51-a88e-c0fc49052ab1","Type":"ContainerStarted","Data":"d45aa4bd4f858534ed86f98dc5fb1c018e4ac8f2570751314dc91252c0723279"}
Oct 30 19:43:47.377712 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:47.372909    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4a4e2f-vp8vp" event={"ID":"80b91a64-9588-4a51-a88e-c0fc49052ab1","Type":"ContainerStarted","Data":"e38ea11bd2ece1b6c09d0db0d9fe8b5d960c5afb63a9b881a4ec1286255be7b6"}
Oct 30 19:43:47.390446 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:47.389919    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-28885407-b2tx8"
Oct 30 19:43:47.560062 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:43:47.559781    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:43:47.595084 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:47.595056    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-dd23d1-mngxw"
Oct 30 19:43:47.643030 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:47.642929    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4a4e2f-vp8vp" podStartSLOduration=1.642914228 podStartE2EDuration="1.642914228s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:43:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:43:47.439199763 +0000 UTC m=+1339.282131702" watchObservedRunningTime="2026-10-30 19:43:47.642914228 +0000 UTC m=+1339.485846160"
Oct 30 19:43:47.707067 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:47.707016    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9e9a9805-721e-4939-a8c4-d078d6a3dc32-kube-api-access-zh8ch\" (UniqueName: \"kubernetes.io/projected/9e9a9805-721e-4939-a8c4-d078d6a3dc32-kube-api-access-zh8ch\") pod \"9e9a9805-721e-4939-a8c4-d078d6a3dc32\" (UID: \"9e9a9805-721e-4939-a8c4-d078d6a3dc32\") "
Oct 30 19:43:47.707242 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:47.707193    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9e9a9805-721e-4939-a8c4-d078d6a3dc32-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9e9a9805-721e-4939-a8c4-d078d6a3dc32-maas-api-openshift-ca\") pod \"9e9a9805-721e-4939-a8c4-d078d6a3dc32\" (UID: \"9e9a9805-721e-4939-a8c4-d078d6a3dc32\") "
Oct 30 19:43:47.707305 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:47.707249    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9e9a9805-721e-4939-a8c4-d078d6a3dc32-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9e9a9805-721e-4939-a8c4-d078d6a3dc32-maas-api-xks-ca\") pod \"9e9a9805-721e-4939-a8c4-d078d6a3dc32\" (UID: \"9e9a9805-721e-4939-a8c4-d078d6a3dc32\") "
Oct 30 19:43:47.707557 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:47.707532    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e9a9805-721e-4939-a8c4-d078d6a3dc32-maas-api-openshift-ca" pod "9e9a9805-721e-4939-a8c4-d078d6a3dc32" (UID: "9e9a9805-721e-4939-a8c4-d078d6a3dc32"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:47.707961 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:47.707942    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9e9a9805-721e-4939-a8c4-d078d6a3dc32-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:47.709312 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:47.709291    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e9a9805-721e-4939-a8c4-d078d6a3dc32-kube-api-access-zh8ch" pod "9e9a9805-721e-4939-a8c4-d078d6a3dc32" (UID: "9e9a9805-721e-4939-a8c4-d078d6a3dc32"). InnerVolumeSpecName "kube-api-access-zh8ch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:47.709493 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:47.709470    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e9a9805-721e-4939-a8c4-d078d6a3dc32-maas-api-xks-ca" pod "9e9a9805-721e-4939-a8c4-d078d6a3dc32" (UID: "9e9a9805-721e-4939-a8c4-d078d6a3dc32"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:43:47.761927 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:47.761895    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-28885407-b2tx8"]
Oct 30 19:43:47.762080 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:43:47.762053    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb6373d5_f4d9_4110_b9f9_2e1968021aee.slice/crio-aa25437e770edbefe636e1512c63c81eb5f63d79d2d2284b57a8e7d92a67fe7e WatchSource:0}: Error finding container aa25437e770edbefe636e1512c63c81eb5f63d79d2d2284b57a8e7d92a67fe7e: Status 404 returned error can't find the container with id aa25437e770edbefe636e1512c63c81eb5f63d79d2d2284b57a8e7d92a67fe7e
Oct 30 19:43:47.809766 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:47.809743    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9e9a9805-721e-4939-a8c4-d078d6a3dc32-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:47.809766 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:47.809766    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zh8ch\" (UniqueName: \"kubernetes.io/projected/9e9a9805-721e-4939-a8c4-d078d6a3dc32-kube-api-access-zh8ch\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:48.294435 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:48.294355    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-3add95-5f665b5b94-hdzl8"
Oct 30 19:43:48.295211 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:48.295191    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3add95-d9bb45d9f-sjmpj"
Oct 30 19:43:48.384512 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:48.384472    5907 generic.go:356] "Generic (PLEG): container finished" podID="80b91a64-9588-4a51-a88e-c0fc49052ab1" containerID="d45aa4bd4f858534ed86f98dc5fb1c018e4ac8f2570751314dc91252c0723279" exitCode=0
Oct 30 19:43:48.384760 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:48.384511    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4a4e2f-vp8vp" event={"ID":"80b91a64-9588-4a51-a88e-c0fc49052ab1","Type":"ContainerDied","Data":"d45aa4bd4f858534ed86f98dc5fb1c018e4ac8f2570751314dc91252c0723279"}
Oct 30 19:43:48.387401 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:48.387349    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-28885407-b2tx8" event={"ID":"eb6373d5-f4d9-4110-b9f9-2e1968021aee","Type":"ContainerStarted","Data":"6ffe00e0be15c029e66dcd280e846667d022b2fc78e9dae6b26e63773559be0d"}
Oct 30 19:43:48.387550 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:48.387429    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-28885407-b2tx8" event={"ID":"eb6373d5-f4d9-4110-b9f9-2e1968021aee","Type":"ContainerStarted","Data":"aa25437e770edbefe636e1512c63c81eb5f63d79d2d2284b57a8e7d92a67fe7e"}
Oct 30 19:43:48.389428 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:48.389405    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-dd23d1-mngxw"
Oct 30 19:43:48.390540 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:48.389403    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-dd23d1-mngxw" event={"ID":"9e9a9805-721e-4939-a8c4-d078d6a3dc32","Type":"ContainerDied","Data":"5ecc0eb6d0bcbaf872c3a2674ea16127551c74645b0837236c3609e3742f4ab3"}
Oct 30 19:43:48.390705 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:48.390515    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5ecc0eb6d0bcbaf872c3a2674ea16127551c74645b0837236c3609e3742f4ab3"
Oct 30 19:43:48.523685 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:48.523591    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-28885407-b2tx8" podStartSLOduration=2.523569801 podStartE2EDuration="2.523569801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:43:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:43:48.501433914 +0000 UTC m=+1340.344365853" watchObservedRunningTime="2026-10-30 19:43:48.523569801 +0000 UTC m=+1340.366501744"
Oct 30 19:43:49.220010 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.219983    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:43:49.245954 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.245924    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-istiod-ca-cert\") pod \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") "
Oct 30 19:43:49.246138 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.245983    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-credential-socket\") pod \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") "
Oct 30 19:43:49.246138 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.246025    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-workload-certs\") pod \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") "
Oct 30 19:43:49.246138 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.246080    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-kube-api-access-7dr8b\" (UniqueName: \"kubernetes.io/projected/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-kube-api-access-7dr8b\") pod \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") "
Oct 30 19:43:49.246323 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.246150    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-istio-token\" (UniqueName: \"kubernetes.io/projected/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-istio-token\") pod \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") "
Oct 30 19:43:49.246323 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.246257    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-istio-podinfo\") pod \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") "
Oct 30 19:43:49.246323 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.246286    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-workload-socket\") pod \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") "
Oct 30 19:43:49.246490 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.246335    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-istio-data\") pod \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") "
Oct 30 19:43:49.246490 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.246344    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-credential-socket" pod "29fbb7be-9068-40ec-a0d4-9fa0ed0469bf" (UID: "29fbb7be-9068-40ec-a0d4-9fa0ed0469bf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:49.246490 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.246356    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-workload-certs" pod "29fbb7be-9068-40ec-a0d4-9fa0ed0469bf" (UID: "29fbb7be-9068-40ec-a0d4-9fa0ed0469bf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:49.246490 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.246381    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-istiod-ca-cert" pod "29fbb7be-9068-40ec-a0d4-9fa0ed0469bf" (UID: "29fbb7be-9068-40ec-a0d4-9fa0ed0469bf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:49.246490 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.246463    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-istio-envoy\") pod \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\" (UID: \"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf\") "
Oct 30 19:43:49.246729 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.246586    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-istio-data" pod "29fbb7be-9068-40ec-a0d4-9fa0ed0469bf" (UID: "29fbb7be-9068-40ec-a0d4-9fa0ed0469bf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:49.246729 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.246599    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-workload-socket" pod "29fbb7be-9068-40ec-a0d4-9fa0ed0469bf" (UID: "29fbb7be-9068-40ec-a0d4-9fa0ed0469bf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:49.247250 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.247230    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:49.247360 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.247253    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:49.247360 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.247267    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:49.247360 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.247279    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:49.247360 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.247291    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:49.248650 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.248626    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-istio-podinfo" pod "29fbb7be-9068-40ec-a0d4-9fa0ed0469bf" (UID: "29fbb7be-9068-40ec-a0d4-9fa0ed0469bf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:43:49.248784 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.248762    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-istio-envoy" pod "29fbb7be-9068-40ec-a0d4-9fa0ed0469bf" (UID: "29fbb7be-9068-40ec-a0d4-9fa0ed0469bf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:49.248933 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.248914    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-istio-token" pod "29fbb7be-9068-40ec-a0d4-9fa0ed0469bf" (UID: "29fbb7be-9068-40ec-a0d4-9fa0ed0469bf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:49.248977 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.248942    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-kube-api-access-7dr8b" pod "29fbb7be-9068-40ec-a0d4-9fa0ed0469bf" (UID: "29fbb7be-9068-40ec-a0d4-9fa0ed0469bf"). InnerVolumeSpecName "kube-api-access-7dr8b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:49.348801 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.348650    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dr8b\" (UniqueName: \"kubernetes.io/projected/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-kube-api-access-7dr8b\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:49.348801 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.348690    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:49.348801 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.348705    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:49.348801 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.348716    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:49.403823 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.402898    5907 generic.go:356] "Generic (PLEG): container finished" podID="eb6373d5-f4d9-4110-b9f9-2e1968021aee" containerID="6ffe00e0be15c029e66dcd280e846667d022b2fc78e9dae6b26e63773559be0d" exitCode=0
Oct 30 19:43:49.403823 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.403092    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-28885407-b2tx8" event={"ID":"eb6373d5-f4d9-4110-b9f9-2e1968021aee","Type":"ContainerDied","Data":"6ffe00e0be15c029e66dcd280e846667d022b2fc78e9dae6b26e63773559be0d"}
Oct 30 19:43:49.406362 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.406333    5907 generic.go:356] "Generic (PLEG): container finished" podID="29fbb7be-9068-40ec-a0d4-9fa0ed0469bf" containerID="7d3b1f55131c855d23bf3ff42421397abc230221ce98296c182f1e26ace045d6" exitCode=0
Oct 30 19:43:49.406506 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.406390    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"
Oct 30 19:43:49.406506 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.406420    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp" event={"ID":"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf","Type":"ContainerDied","Data":"7d3b1f55131c855d23bf3ff42421397abc230221ce98296c182f1e26ace045d6"}
Oct 30 19:43:49.406506 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.406472    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp" event={"ID":"29fbb7be-9068-40ec-a0d4-9fa0ed0469bf","Type":"ContainerDied","Data":"7f5eb2bd78a58c7aae9a03ba8b36c0509706c243c18268ba3fa1c396e7cd634d"}
Oct 30 19:43:49.406506 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.406501    5907 scope.go:122] "RemoveContainer" containerID="7d3b1f55131c855d23bf3ff42421397abc230221ce98296c182f1e26ace045d6"
Oct 30 19:43:49.419277 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.419254    5907 scope.go:122] "RemoveContainer" containerID="e3a35def6621ba550ad37eeb67feec3899966bcf15d06ef4badc081f7d5849eb"
Oct 30 19:43:49.431075 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.431050    5907 scope.go:122] "RemoveContainer" containerID="7d3b1f55131c855d23bf3ff42421397abc230221ce98296c182f1e26ace045d6"
Oct 30 19:43:49.431438 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:43:49.431401    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d3b1f55131c855d23bf3ff42421397abc230221ce98296c182f1e26ace045d6\": container with ID starting with 7d3b1f55131c855d23bf3ff42421397abc230221ce98296c182f1e26ace045d6 not found: ID does not exist" containerID="7d3b1f55131c855d23bf3ff42421397abc230221ce98296c182f1e26ace045d6"
Oct 30 19:43:49.431507 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.431431    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d3b1f55131c855d23bf3ff42421397abc230221ce98296c182f1e26ace045d6"} err="failed to get container status \"7d3b1f55131c855d23bf3ff42421397abc230221ce98296c182f1e26ace045d6\": rpc error: code = NotFound desc = could not find container \"7d3b1f55131c855d23bf3ff42421397abc230221ce98296c182f1e26ace045d6\": container with ID starting with 7d3b1f55131c855d23bf3ff42421397abc230221ce98296c182f1e26ace045d6 not found: ID does not exist"
Oct 30 19:43:49.431507 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.431456    5907 scope.go:122] "RemoveContainer" containerID="e3a35def6621ba550ad37eeb67feec3899966bcf15d06ef4badc081f7d5849eb"
Oct 30 19:43:49.431770 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:43:49.431752    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3a35def6621ba550ad37eeb67feec3899966bcf15d06ef4badc081f7d5849eb\": container with ID starting with e3a35def6621ba550ad37eeb67feec3899966bcf15d06ef4badc081f7d5849eb not found: ID does not exist" containerID="e3a35def6621ba550ad37eeb67feec3899966bcf15d06ef4badc081f7d5849eb"
Oct 30 19:43:49.431827 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.431775    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3a35def6621ba550ad37eeb67feec3899966bcf15d06ef4badc081f7d5849eb"} err="failed to get container status \"e3a35def6621ba550ad37eeb67feec3899966bcf15d06ef4badc081f7d5849eb\": rpc error: code = NotFound desc = could not find container \"e3a35def6621ba550ad37eeb67feec3899966bcf15d06ef4badc081f7d5849eb\": container with ID starting with e3a35def6621ba550ad37eeb67feec3899966bcf15d06ef4badc081f7d5849eb not found: ID does not exist"
Oct 30 19:43:49.521777 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.516021    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"]
Oct 30 19:43:49.529279 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.529242    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-7e89da-openshift-default-58546bcb99-x5fnp"]
Oct 30 19:43:49.596256 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.596227    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4a4e2f-vp8vp"
Oct 30 19:43:49.654259 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.654225    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/80b91a64-9588-4a51-a88e-c0fc49052ab1-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/80b91a64-9588-4a51-a88e-c0fc49052ab1-maas-api-xks-ca\") pod \"80b91a64-9588-4a51-a88e-c0fc49052ab1\" (UID: \"80b91a64-9588-4a51-a88e-c0fc49052ab1\") "
Oct 30 19:43:49.654441 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.654275    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/80b91a64-9588-4a51-a88e-c0fc49052ab1-kube-api-access-6bmxt\" (UniqueName: \"kubernetes.io/projected/80b91a64-9588-4a51-a88e-c0fc49052ab1-kube-api-access-6bmxt\") pod \"80b91a64-9588-4a51-a88e-c0fc49052ab1\" (UID: \"80b91a64-9588-4a51-a88e-c0fc49052ab1\") "
Oct 30 19:43:49.654441 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.654351    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/80b91a64-9588-4a51-a88e-c0fc49052ab1-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/80b91a64-9588-4a51-a88e-c0fc49052ab1-maas-api-openshift-ca\") pod \"80b91a64-9588-4a51-a88e-c0fc49052ab1\" (UID: \"80b91a64-9588-4a51-a88e-c0fc49052ab1\") "
Oct 30 19:43:49.654828 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.654784    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80b91a64-9588-4a51-a88e-c0fc49052ab1-maas-api-openshift-ca" pod "80b91a64-9588-4a51-a88e-c0fc49052ab1" (UID: "80b91a64-9588-4a51-a88e-c0fc49052ab1"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:49.658099 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.658064    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80b91a64-9588-4a51-a88e-c0fc49052ab1-maas-api-xks-ca" pod "80b91a64-9588-4a51-a88e-c0fc49052ab1" (UID: "80b91a64-9588-4a51-a88e-c0fc49052ab1"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:43:49.658099 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.658084    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80b91a64-9588-4a51-a88e-c0fc49052ab1-kube-api-access-6bmxt" pod "80b91a64-9588-4a51-a88e-c0fc49052ab1" (UID: "80b91a64-9588-4a51-a88e-c0fc49052ab1"). InnerVolumeSpecName "kube-api-access-6bmxt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:49.658588 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.658547    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/80b91a64-9588-4a51-a88e-c0fc49052ab1-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:49.658629 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.658597    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6bmxt\" (UniqueName: \"kubernetes.io/projected/80b91a64-9588-4a51-a88e-c0fc49052ab1-kube-api-access-6bmxt\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:49.658629 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:49.658615    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/80b91a64-9588-4a51-a88e-c0fc49052ab1-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:50.135888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:50.120214    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:43:50.361218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:50.361178    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="29fbb7be-9068-40ec-a0d4-9fa0ed0469bf" path="/var/lib/kubelet/pods/29fbb7be-9068-40ec-a0d4-9fa0ed0469bf/volumes"
Oct 30 19:43:50.415564 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:50.415475    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4a4e2f-vp8vp"
Oct 30 19:43:50.415564 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:50.415484    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4a4e2f-vp8vp" event={"ID":"80b91a64-9588-4a51-a88e-c0fc49052ab1","Type":"ContainerDied","Data":"e38ea11bd2ece1b6c09d0db0d9fe8b5d960c5afb63a9b881a4ec1286255be7b6"}
Oct 30 19:43:50.415564 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:50.415533    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e38ea11bd2ece1b6c09d0db0d9fe8b5d960c5afb63a9b881a4ec1286255be7b6"
Oct 30 19:43:50.619392 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:50.619369    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-28885407-b2tx8"
Oct 30 19:43:50.677734 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:50.677649    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb6373d5-f4d9-4110-b9f9-2e1968021aee-kube-api-access-5b6b4\" (UniqueName: \"kubernetes.io/projected/eb6373d5-f4d9-4110-b9f9-2e1968021aee-kube-api-access-5b6b4\") pod \"eb6373d5-f4d9-4110-b9f9-2e1968021aee\" (UID: \"eb6373d5-f4d9-4110-b9f9-2e1968021aee\") "
Oct 30 19:43:50.677916 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:50.677836    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/eb6373d5-f4d9-4110-b9f9-2e1968021aee-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/eb6373d5-f4d9-4110-b9f9-2e1968021aee-maas-api-xks-ca\") pod \"eb6373d5-f4d9-4110-b9f9-2e1968021aee\" (UID: \"eb6373d5-f4d9-4110-b9f9-2e1968021aee\") "
Oct 30 19:43:50.678158 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:50.678134    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eb6373d5-f4d9-4110-b9f9-2e1968021aee-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/eb6373d5-f4d9-4110-b9f9-2e1968021aee-maas-api-openshift-ca\") pod \"eb6373d5-f4d9-4110-b9f9-2e1968021aee\" (UID: \"eb6373d5-f4d9-4110-b9f9-2e1968021aee\") "
Oct 30 19:43:50.679462 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:50.679436    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb6373d5-f4d9-4110-b9f9-2e1968021aee-maas-api-openshift-ca" pod "eb6373d5-f4d9-4110-b9f9-2e1968021aee" (UID: "eb6373d5-f4d9-4110-b9f9-2e1968021aee"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:50.688766 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:50.688741    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb6373d5-f4d9-4110-b9f9-2e1968021aee-kube-api-access-5b6b4" pod "eb6373d5-f4d9-4110-b9f9-2e1968021aee" (UID: "eb6373d5-f4d9-4110-b9f9-2e1968021aee"). InnerVolumeSpecName "kube-api-access-5b6b4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:50.690635 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:50.690602    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb6373d5-f4d9-4110-b9f9-2e1968021aee-maas-api-xks-ca" pod "eb6373d5-f4d9-4110-b9f9-2e1968021aee" (UID: "eb6373d5-f4d9-4110-b9f9-2e1968021aee"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:43:50.781074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:50.781042    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/eb6373d5-f4d9-4110-b9f9-2e1968021aee-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:50.781074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:50.781072    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5b6b4\" (UniqueName: \"kubernetes.io/projected/eb6373d5-f4d9-4110-b9f9-2e1968021aee-kube-api-access-5b6b4\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:50.781251 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:50.781085    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/eb6373d5-f4d9-4110-b9f9-2e1968021aee-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:51.428292 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:51.428242    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-28885407-b2tx8"
Oct 30 19:43:51.428799 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:51.428231    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-28885407-b2tx8" event={"ID":"eb6373d5-f4d9-4110-b9f9-2e1968021aee","Type":"ContainerDied","Data":"aa25437e770edbefe636e1512c63c81eb5f63d79d2d2284b57a8e7d92a67fe7e"}
Oct 30 19:43:51.428799 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:51.428367    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aa25437e770edbefe636e1512c63c81eb5f63d79d2d2284b57a8e7d92a67fe7e"
Oct 30 19:43:52.586891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.571083    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"
Oct 30 19:43:52.618960 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.616373    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71285665-ae23-4783-a24f-535dccfb68c7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71285665-ae23-4783-a24f-535dccfb68c7-istio-data\") pod \"71285665-ae23-4783-a24f-535dccfb68c7\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") "
Oct 30 19:43:52.618960 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.616459    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/71285665-ae23-4783-a24f-535dccfb68c7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71285665-ae23-4783-a24f-535dccfb68c7-istio-podinfo\") pod \"71285665-ae23-4783-a24f-535dccfb68c7\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") "
Oct 30 19:43:52.618960 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.616587    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71285665-ae23-4783-a24f-535dccfb68c7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71285665-ae23-4783-a24f-535dccfb68c7-workload-socket\") pod \"71285665-ae23-4783-a24f-535dccfb68c7\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") "
Oct 30 19:43:52.618960 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.616638    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/71285665-ae23-4783-a24f-535dccfb68c7-kube-api-access-xnxsg\" (UniqueName: \"kubernetes.io/projected/71285665-ae23-4783-a24f-535dccfb68c7-kube-api-access-xnxsg\") pod \"71285665-ae23-4783-a24f-535dccfb68c7\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") "
Oct 30 19:43:52.618960 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.616732    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71285665-ae23-4783-a24f-535dccfb68c7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71285665-ae23-4783-a24f-535dccfb68c7-istio-envoy\") pod \"71285665-ae23-4783-a24f-535dccfb68c7\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") "
Oct 30 19:43:52.618960 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.616799    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71285665-ae23-4783-a24f-535dccfb68c7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71285665-ae23-4783-a24f-535dccfb68c7-credential-socket\") pod \"71285665-ae23-4783-a24f-535dccfb68c7\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") "
Oct 30 19:43:52.618960 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.616829    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71285665-ae23-4783-a24f-535dccfb68c7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71285665-ae23-4783-a24f-535dccfb68c7-workload-certs\") pod \"71285665-ae23-4783-a24f-535dccfb68c7\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") "
Oct 30 19:43:52.618960 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.616901    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/71285665-ae23-4783-a24f-535dccfb68c7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71285665-ae23-4783-a24f-535dccfb68c7-istiod-ca-cert\") pod \"71285665-ae23-4783-a24f-535dccfb68c7\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") "
Oct 30 19:43:52.618960 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.617054    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/71285665-ae23-4783-a24f-535dccfb68c7-istio-token\" (UniqueName: \"kubernetes.io/projected/71285665-ae23-4783-a24f-535dccfb68c7-istio-token\") pod \"71285665-ae23-4783-a24f-535dccfb68c7\" (UID: \"71285665-ae23-4783-a24f-535dccfb68c7\") "
Oct 30 19:43:52.674216 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.647319    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71285665-ae23-4783-a24f-535dccfb68c7-kube-api-access-xnxsg" pod "71285665-ae23-4783-a24f-535dccfb68c7" (UID: "71285665-ae23-4783-a24f-535dccfb68c7"). InnerVolumeSpecName "kube-api-access-xnxsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:52.674216 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.647583    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71285665-ae23-4783-a24f-535dccfb68c7-istio-data" pod "71285665-ae23-4783-a24f-535dccfb68c7" (UID: "71285665-ae23-4783-a24f-535dccfb68c7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:52.681400 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.681099    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71285665-ae23-4783-a24f-535dccfb68c7-workload-socket" pod "71285665-ae23-4783-a24f-535dccfb68c7" (UID: "71285665-ae23-4783-a24f-535dccfb68c7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:52.681676 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.681637    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71285665-ae23-4783-a24f-535dccfb68c7-workload-certs" pod "71285665-ae23-4783-a24f-535dccfb68c7" (UID: "71285665-ae23-4783-a24f-535dccfb68c7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:52.687517 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.687487    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71285665-ae23-4783-a24f-535dccfb68c7-istiod-ca-cert" pod "71285665-ae23-4783-a24f-535dccfb68c7" (UID: "71285665-ae23-4783-a24f-535dccfb68c7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:52.704110 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.690111    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71285665-ae23-4783-a24f-535dccfb68c7-credential-socket" pod "71285665-ae23-4783-a24f-535dccfb68c7" (UID: "71285665-ae23-4783-a24f-535dccfb68c7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:52.719444 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.719414    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71285665-ae23-4783-a24f-535dccfb68c7-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:52.719679 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.719662    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnxsg\" (UniqueName: \"kubernetes.io/projected/71285665-ae23-4783-a24f-535dccfb68c7-kube-api-access-xnxsg\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:52.719786 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.719774    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71285665-ae23-4783-a24f-535dccfb68c7-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:52.719901 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.719887    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71285665-ae23-4783-a24f-535dccfb68c7-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:52.732962 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.732924    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71285665-ae23-4783-a24f-535dccfb68c7-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:52.733175 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.733161    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71285665-ae23-4783-a24f-535dccfb68c7-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:52.755907 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.754589    5907 generic.go:356] "Generic (PLEG): container finished" podID="01992c2f-0e71-4b86-8e8a-69bb7727bd6a" containerID="517f1f95ec7893dc438e29d7c93505719922128a86c72f062d50868d00705195" exitCode=0
Oct 30 19:43:52.755907 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.754687    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv" event={"ID":"01992c2f-0e71-4b86-8e8a-69bb7727bd6a","Type":"ContainerDied","Data":"517f1f95ec7893dc438e29d7c93505719922128a86c72f062d50868d00705195"}
Oct 30 19:43:52.780908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.772688    5907 generic.go:356] "Generic (PLEG): container finished" podID="71285665-ae23-4783-a24f-535dccfb68c7" containerID="7e0b09bd4ae355d8392bff6d6599fdfe47e0360a686924bf9de3595f210fff14" exitCode=0
Oct 30 19:43:52.780908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.772902    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np" event={"ID":"71285665-ae23-4783-a24f-535dccfb68c7","Type":"ContainerDied","Data":"7e0b09bd4ae355d8392bff6d6599fdfe47e0360a686924bf9de3595f210fff14"}
Oct 30 19:43:52.780908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.772963    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np" event={"ID":"71285665-ae23-4783-a24f-535dccfb68c7","Type":"ContainerDied","Data":"426dfbe93bb9bf3f6116209daa2db778603675f13cb0ffa12aefadfa22f6669a"}
Oct 30 19:43:52.780908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.772996    5907 scope.go:122] "RemoveContainer" containerID="7e0b09bd4ae355d8392bff6d6599fdfe47e0360a686924bf9de3595f210fff14"
Oct 30 19:43:52.780908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.773160    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"
Oct 30 19:43:52.818410 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.785987    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71285665-ae23-4783-a24f-535dccfb68c7-istio-envoy" pod "71285665-ae23-4783-a24f-535dccfb68c7" (UID: "71285665-ae23-4783-a24f-535dccfb68c7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:52.818410 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.788219    5907 generic.go:356] "Generic (PLEG): container finished" podID="b6947d3d-3c9a-40d6-a9ae-0604ec1c7125" containerID="e7c613909fa4ea2bd315211f18eb37a63655bee217949710516c09dd1ff9a3fa" exitCode=0
Oct 30 19:43:52.818410 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.788299    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq" event={"ID":"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125","Type":"ContainerDied","Data":"e7c613909fa4ea2bd315211f18eb37a63655bee217949710516c09dd1ff9a3fa"}
Oct 30 19:43:52.818410 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.790602    5907 generic.go:356] "Generic (PLEG): container finished" podID="27e1950d-b453-4f32-afd2-95ab9027a2e5" containerID="6ef915f651c8f45e0b04b63b9bb1e6c7d25192c102ac8f92036f1cea6a57e038" exitCode=0
Oct 30 19:43:52.818410 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.790754    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4" event={"ID":"27e1950d-b453-4f32-afd2-95ab9027a2e5","Type":"ContainerDied","Data":"6ef915f651c8f45e0b04b63b9bb1e6c7d25192c102ac8f92036f1cea6a57e038"}
Oct 30 19:43:52.858683 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.831214    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/71285665-ae23-4783-a24f-535dccfb68c7-istio-podinfo" pod "71285665-ae23-4783-a24f-535dccfb68c7" (UID: "71285665-ae23-4783-a24f-535dccfb68c7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:43:52.858683 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.835578    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71285665-ae23-4783-a24f-535dccfb68c7-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:52.858683 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:52.835604    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71285665-ae23-4783-a24f-535dccfb68c7-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:53.205439 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:53.205341    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71285665-ae23-4783-a24f-535dccfb68c7-istio-token" pod "71285665-ae23-4783-a24f-535dccfb68c7" (UID: "71285665-ae23-4783-a24f-535dccfb68c7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:53.263000 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:53.261450    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71285665-ae23-4783-a24f-535dccfb68c7-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:53.331702 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:53.330501    5907 scope.go:122] "RemoveContainer" containerID="7e0b09bd4ae355d8392bff6d6599fdfe47e0360a686924bf9de3595f210fff14"
Oct 30 19:43:53.332066 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:43:53.331931    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e0b09bd4ae355d8392bff6d6599fdfe47e0360a686924bf9de3595f210fff14\": container with ID starting with 7e0b09bd4ae355d8392bff6d6599fdfe47e0360a686924bf9de3595f210fff14 not found: ID does not exist" containerID="7e0b09bd4ae355d8392bff6d6599fdfe47e0360a686924bf9de3595f210fff14"
Oct 30 19:43:53.332066 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:53.332009    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e0b09bd4ae355d8392bff6d6599fdfe47e0360a686924bf9de3595f210fff14"} err="failed to get container status \"7e0b09bd4ae355d8392bff6d6599fdfe47e0360a686924bf9de3595f210fff14\": rpc error: code = NotFound desc = could not find container \"7e0b09bd4ae355d8392bff6d6599fdfe47e0360a686924bf9de3595f210fff14\": container with ID starting with 7e0b09bd4ae355d8392bff6d6599fdfe47e0360a686924bf9de3595f210fff14 not found: ID does not exist"
Oct 30 19:43:53.476889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:53.472557    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"
Oct 30 19:43:53.489603 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:53.488191    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"
Oct 30 19:43:53.490822 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:53.490796    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"
Oct 30 19:43:53.745898 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:53.743335    5907 patch_prober.go:29] interesting pod/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:43:53.745898 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:53.743393    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv" podUID="01992c2f-0e71-4b86-8e8a-69bb7727bd6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:43:54.133098 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.133039    5907 patch_prober.go:29] interesting pod/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:43:54.139928 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.133424    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq" podUID="b6947d3d-3c9a-40d6-a9ae-0604ec1c7125" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:43:54.347886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.335046    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/27e1950d-b453-4f32-afd2-95ab9027a2e5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27e1950d-b453-4f32-afd2-95ab9027a2e5-istiod-ca-cert\") pod \"27e1950d-b453-4f32-afd2-95ab9027a2e5\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") "
Oct 30 19:43:54.347886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.335116    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istio-token\" (UniqueName: \"kubernetes.io/projected/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istio-token\") pod \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") "
Oct 30 19:43:54.347886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.335166    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/27e1950d-b453-4f32-afd2-95ab9027a2e5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27e1950d-b453-4f32-afd2-95ab9027a2e5-istio-data\") pod \"27e1950d-b453-4f32-afd2-95ab9027a2e5\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") "
Oct 30 19:43:54.347886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.335238    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istio-podinfo\") pod \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") "
Oct 30 19:43:54.347886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.335279    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/27e1950d-b453-4f32-afd2-95ab9027a2e5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27e1950d-b453-4f32-afd2-95ab9027a2e5-istio-envoy\") pod \"27e1950d-b453-4f32-afd2-95ab9027a2e5\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") "
Oct 30 19:43:54.347886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.335327    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/27e1950d-b453-4f32-afd2-95ab9027a2e5-kube-api-access-gcllf\" (UniqueName: \"kubernetes.io/projected/27e1950d-b453-4f32-afd2-95ab9027a2e5-kube-api-access-gcllf\") pod \"27e1950d-b453-4f32-afd2-95ab9027a2e5\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") "
Oct 30 19:43:54.347886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.335447    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istiod-ca-cert\") pod \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") "
Oct 30 19:43:54.347886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.335480    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/27e1950d-b453-4f32-afd2-95ab9027a2e5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27e1950d-b453-4f32-afd2-95ab9027a2e5-workload-socket\") pod \"27e1950d-b453-4f32-afd2-95ab9027a2e5\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") "
Oct 30 19:43:54.347886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.335526    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istio-data\") pod \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") "
Oct 30 19:43:54.347886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.335578    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/27e1950d-b453-4f32-afd2-95ab9027a2e5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27e1950d-b453-4f32-afd2-95ab9027a2e5-istio-podinfo\") pod \"27e1950d-b453-4f32-afd2-95ab9027a2e5\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") "
Oct 30 19:43:54.347886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.335633    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/27e1950d-b453-4f32-afd2-95ab9027a2e5-istio-token\" (UniqueName: \"kubernetes.io/projected/27e1950d-b453-4f32-afd2-95ab9027a2e5-istio-token\") pod \"27e1950d-b453-4f32-afd2-95ab9027a2e5\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") "
Oct 30 19:43:54.347886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.335675    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-kube-api-access-p48gn\" (UniqueName: \"kubernetes.io/projected/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-kube-api-access-p48gn\") pod \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") "
Oct 30 19:43:54.347886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.335725    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-kube-api-access-sc9tr\" (UniqueName: \"kubernetes.io/projected/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-kube-api-access-sc9tr\") pod \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") "
Oct 30 19:43:54.347886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.335787    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istio-data\") pod \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") "
Oct 30 19:43:54.347886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.335889    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istio-envoy\") pod \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") "
Oct 30 19:43:54.347886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.335929    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-workload-socket\") pod \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") "
Oct 30 19:43:54.347886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.336020    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-credential-socket\") pod \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") "
Oct 30 19:43:54.347886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.336057    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istio-token\" (UniqueName: \"kubernetes.io/projected/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istio-token\") pod \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") "
Oct 30 19:43:54.347886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.336108    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/27e1950d-b453-4f32-afd2-95ab9027a2e5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27e1950d-b453-4f32-afd2-95ab9027a2e5-credential-socket\") pod \"27e1950d-b453-4f32-afd2-95ab9027a2e5\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") "
Oct 30 19:43:54.347886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.336185    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-workload-certs\") pod \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") "
Oct 30 19:43:54.347886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.336223    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-workload-socket\") pod \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") "
Oct 30 19:43:54.347886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.336296    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istiod-ca-cert\") pod \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") "
Oct 30 19:43:54.347886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.336346    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istio-envoy\") pod \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") "
Oct 30 19:43:54.347886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.336410    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istio-podinfo\") pod \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") "
Oct 30 19:43:54.347886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.336471    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/27e1950d-b453-4f32-afd2-95ab9027a2e5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27e1950d-b453-4f32-afd2-95ab9027a2e5-workload-certs\") pod \"27e1950d-b453-4f32-afd2-95ab9027a2e5\" (UID: \"27e1950d-b453-4f32-afd2-95ab9027a2e5\") "
Oct 30 19:43:54.347886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.336592    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-credential-socket\") pod \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") "
Oct 30 19:43:54.347886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.336668    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-workload-certs\") pod \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") "
Oct 30 19:43:54.541171 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.541073    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-workload-certs" pod "01992c2f-0e71-4b86-8e8a-69bb7727bd6a" (UID: "01992c2f-0e71-4b86-8e8a-69bb7727bd6a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:54.541987 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.541960    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27e1950d-b453-4f32-afd2-95ab9027a2e5-workload-socket" pod "27e1950d-b453-4f32-afd2-95ab9027a2e5" (UID: "27e1950d-b453-4f32-afd2-95ab9027a2e5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:54.562337 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.562272    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27e1950d-b453-4f32-afd2-95ab9027a2e5-workload-certs" pod "27e1950d-b453-4f32-afd2-95ab9027a2e5" (UID: "27e1950d-b453-4f32-afd2-95ab9027a2e5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:54.563386 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.563360    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-workload-socket" pod "b6947d3d-3c9a-40d6-a9ae-0604ec1c7125" (UID: "b6947d3d-3c9a-40d6-a9ae-0604ec1c7125"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:54.563877 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.563841    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-workload-socket" pod "01992c2f-0e71-4b86-8e8a-69bb7727bd6a" (UID: "01992c2f-0e71-4b86-8e8a-69bb7727bd6a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:54.564190 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.564172    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-workload-certs" pod "b6947d3d-3c9a-40d6-a9ae-0604ec1c7125" (UID: "b6947d3d-3c9a-40d6-a9ae-0604ec1c7125"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:54.617953 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.617908    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27e1950d-b453-4f32-afd2-95ab9027a2e5-istio-token" pod "27e1950d-b453-4f32-afd2-95ab9027a2e5" (UID: "27e1950d-b453-4f32-afd2-95ab9027a2e5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:54.641555 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.641523    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istio-token" pod "b6947d3d-3c9a-40d6-a9ae-0604ec1c7125" (UID: "b6947d3d-3c9a-40d6-a9ae-0604ec1c7125"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:54.643210 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.643191    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istio-data" pod "01992c2f-0e71-4b86-8e8a-69bb7727bd6a" (UID: "01992c2f-0e71-4b86-8e8a-69bb7727bd6a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:54.668991 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.668109    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istio-data\") pod \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\" (UID: \"01992c2f-0e71-4b86-8e8a-69bb7727bd6a\") "
Oct 30 19:43:54.668991 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.668259    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istio-token\" (UniqueName: \"kubernetes.io/projected/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istio-token\") pod \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\" (UID: \"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125\") "
Oct 30 19:43:54.668991 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:43:54.668970    5907 empty_dir.go:505] Warning: Unmount skipped because path does not exist: /var/lib/kubelet/pods/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125/volumes/kubernetes.io~projected/istio-token
Oct 30 19:43:54.669337 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.669027    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istio-token" pod "b6947d3d-3c9a-40d6-a9ae-0604ec1c7125" (UID: "b6947d3d-3c9a-40d6-a9ae-0604ec1c7125"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:54.669337 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:43:54.669077    5907 empty_dir.go:505] Warning: Unmount skipped because path does not exist: /var/lib/kubelet/pods/01992c2f-0e71-4b86-8e8a-69bb7727bd6a/volumes/kubernetes.io~empty-dir/istio-data
Oct 30 19:43:54.669337 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.669086    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istio-data" pod "01992c2f-0e71-4b86-8e8a-69bb7727bd6a" (UID: "01992c2f-0e71-4b86-8e8a-69bb7727bd6a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:54.669645 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.669622    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:54.669645 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.669651    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:54.669785 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.669665    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:54.669785 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.669678    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:54.669785 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.669691    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:54.669785 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.669704    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27e1950d-b453-4f32-afd2-95ab9027a2e5-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:54.669785 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.669717    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:54.669785 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.669731    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27e1950d-b453-4f32-afd2-95ab9027a2e5-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:54.669785 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.669743    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27e1950d-b453-4f32-afd2-95ab9027a2e5-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:54.671760 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.671494    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istiod-ca-cert" pod "01992c2f-0e71-4b86-8e8a-69bb7727bd6a" (UID: "01992c2f-0e71-4b86-8e8a-69bb7727bd6a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:54.703079 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.695208    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istio-podinfo" pod "01992c2f-0e71-4b86-8e8a-69bb7727bd6a" (UID: "01992c2f-0e71-4b86-8e8a-69bb7727bd6a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:43:54.710034 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.706690    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-credential-socket" pod "01992c2f-0e71-4b86-8e8a-69bb7727bd6a" (UID: "01992c2f-0e71-4b86-8e8a-69bb7727bd6a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:54.710034 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.709984    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-credential-socket" pod "b6947d3d-3c9a-40d6-a9ae-0604ec1c7125" (UID: "b6947d3d-3c9a-40d6-a9ae-0604ec1c7125"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:54.710281 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.710254    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv" event={"ID":"01992c2f-0e71-4b86-8e8a-69bb7727bd6a","Type":"ContainerDied","Data":"8b7e7664b1f7ccb6bd604f156a66dfbdd2e0913a422c0452a432cc015121a2e9"}
Oct 30 19:43:54.710478 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.710463    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"]
Oct 30 19:43:54.710583 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.710573    5907 scope.go:122] "RemoveContainer" containerID="517f1f95ec7893dc438e29d7c93505719922128a86c72f062d50868d00705195"
Oct 30 19:43:54.710773 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.710751    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"
Oct 30 19:43:54.711533 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.710879    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istiod-ca-cert" pod "b6947d3d-3c9a-40d6-a9ae-0604ec1c7125" (UID: "b6947d3d-3c9a-40d6-a9ae-0604ec1c7125"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:54.711666 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.711303    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istio-data" pod "b6947d3d-3c9a-40d6-a9ae-0604ec1c7125" (UID: "b6947d3d-3c9a-40d6-a9ae-0604ec1c7125"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:54.743973 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.732619    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27e1950d-b453-4f32-afd2-95ab9027a2e5-credential-socket" pod "27e1950d-b453-4f32-afd2-95ab9027a2e5" (UID: "27e1950d-b453-4f32-afd2-95ab9027a2e5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:54.743973 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.734721    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27e1950d-b453-4f32-afd2-95ab9027a2e5-istiod-ca-cert" pod "27e1950d-b453-4f32-afd2-95ab9027a2e5" (UID: "27e1950d-b453-4f32-afd2-95ab9027a2e5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:54.743973 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.478516    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27e1950d-b453-4f32-afd2-95ab9027a2e5-istio-data" pod "27e1950d-b453-4f32-afd2-95ab9027a2e5" (UID: "27e1950d-b453-4f32-afd2-95ab9027a2e5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:54.743973 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.743772    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-kh2np"]
Oct 30 19:43:54.745398 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.744486    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6" podUID="c3f73773-6c2f-47e4-a5a4-c57916f64ac9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.176:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:43:54.789416 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.770380    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq" event={"ID":"b6947d3d-3c9a-40d6-a9ae-0604ec1c7125","Type":"ContainerDied","Data":"a45092f34486be78454dcd7f3a077648995ac2b334aa7e08ca71619b31b8129a"}
Oct 30 19:43:54.789416 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.770526    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"
Oct 30 19:43:54.793933 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.793119    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:54.793933 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.793151    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:54.793933 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.793166    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:54.793933 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.793183    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27e1950d-b453-4f32-afd2-95ab9027a2e5-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:54.793933 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.793198    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:54.793933 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.793212    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:54.793933 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.793224    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:54.793933 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.793239    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27e1950d-b453-4f32-afd2-95ab9027a2e5-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:54.793933 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.793252    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27e1950d-b453-4f32-afd2-95ab9027a2e5-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:54.818755 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.817082    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-kube-api-access-p48gn" pod "01992c2f-0e71-4b86-8e8a-69bb7727bd6a" (UID: "01992c2f-0e71-4b86-8e8a-69bb7727bd6a"). InnerVolumeSpecName "kube-api-access-p48gn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:54.818755 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.817113    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istio-podinfo" pod "b6947d3d-3c9a-40d6-a9ae-0604ec1c7125" (UID: "b6947d3d-3c9a-40d6-a9ae-0604ec1c7125"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:43:54.818755 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.817138    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istio-token" pod "01992c2f-0e71-4b86-8e8a-69bb7727bd6a" (UID: "01992c2f-0e71-4b86-8e8a-69bb7727bd6a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:54.818755 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.817203    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istio-envoy" pod "01992c2f-0e71-4b86-8e8a-69bb7727bd6a" (UID: "01992c2f-0e71-4b86-8e8a-69bb7727bd6a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:54.819446 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.819391    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27e1950d-b453-4f32-afd2-95ab9027a2e5-kube-api-access-gcllf" pod "27e1950d-b453-4f32-afd2-95ab9027a2e5" (UID: "27e1950d-b453-4f32-afd2-95ab9027a2e5"). InnerVolumeSpecName "kube-api-access-gcllf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:54.819732 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.819691    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27e1950d-b453-4f32-afd2-95ab9027a2e5-istio-envoy" pod "27e1950d-b453-4f32-afd2-95ab9027a2e5" (UID: "27e1950d-b453-4f32-afd2-95ab9027a2e5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:54.820005 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.819985    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/27e1950d-b453-4f32-afd2-95ab9027a2e5-istio-podinfo" pod "27e1950d-b453-4f32-afd2-95ab9027a2e5" (UID: "27e1950d-b453-4f32-afd2-95ab9027a2e5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:43:54.836325 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.829961    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4" event={"ID":"27e1950d-b453-4f32-afd2-95ab9027a2e5","Type":"ContainerDied","Data":"4d954e078d0018c60b7ecc546193ae3c180ab53c8dcc0fb05885ac0675234a3e"}
Oct 30 19:43:54.836325 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.829985    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-kube-api-access-sc9tr" pod "b6947d3d-3c9a-40d6-a9ae-0604ec1c7125" (UID: "b6947d3d-3c9a-40d6-a9ae-0604ec1c7125"). InnerVolumeSpecName "kube-api-access-sc9tr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:54.836325 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.830114    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"
Oct 30 19:43:54.858878 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.858809    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istio-envoy" pod "b6947d3d-3c9a-40d6-a9ae-0604ec1c7125" (UID: "b6947d3d-3c9a-40d6-a9ae-0604ec1c7125"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:54.890050 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.888952    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:43:54.934890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.892111    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:43:54.934890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.909372    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:54.934890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.909406    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:54.934890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.909422    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27e1950d-b453-4f32-afd2-95ab9027a2e5-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:54.934890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.909437    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcllf\" (UniqueName: \"kubernetes.io/projected/27e1950d-b453-4f32-afd2-95ab9027a2e5-kube-api-access-gcllf\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:54.934890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.909452    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27e1950d-b453-4f32-afd2-95ab9027a2e5-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:54.934890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.909467    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p48gn\" (UniqueName: \"kubernetes.io/projected/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-kube-api-access-p48gn\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:54.934890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.909483    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sc9tr\" (UniqueName: \"kubernetes.io/projected/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-kube-api-access-sc9tr\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:54.934890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.909500    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01992c2f-0e71-4b86-8e8a-69bb7727bd6a-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:54.934890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:54.909524    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:43:55.023561 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:55.022015    5907 scope.go:122] "RemoveContainer" containerID="e7c613909fa4ea2bd315211f18eb37a63655bee217949710516c09dd1ff9a3fa"
Oct 30 19:43:55.029970 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:55.029933    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:43:55.237911 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:55.224918    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"]
Oct 30 19:43:55.241365 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:55.241328    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-q2gt4"]
Oct 30 19:43:55.263885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:55.259029    5907 scope.go:122] "RemoveContainer" containerID="6ef915f651c8f45e0b04b63b9bb1e6c7d25192c102ac8f92036f1cea6a57e038"
Oct 30 19:43:55.420884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:55.399925    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"]
Oct 30 19:43:55.448508 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:55.448472    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-h7grq"]
Oct 30 19:43:55.746946 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:55.741166    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"]
Oct 30 19:43:55.746946 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:55.744785    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:43:55.746946 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:55.746240    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6" podUID="c3f73773-6c2f-47e4-a5a4-c57916f64ac9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.176:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:43:55.746946 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:55.746836    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"
Oct 30 19:43:55.856137 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:55.856103    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-ef9a5a-openshift-default-7bd7f4b75f-2rjcv"]
Oct 30 19:43:56.016970 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:55.977577    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 19:43:56.016970 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:55.985047    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:43:56.016970 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:56.004983    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:43:56.016970 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:56.005055    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:43:56.031506 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:56.031468    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:43:56.047971 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:56.047938    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:43:56.091801 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:56.089900    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-dd23d1-dddbddc84-htdcq"]
Oct 30 19:43:56.141432 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:56.141368    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:43:56.201482 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:56.194565    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:43:56.931823 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:56.931786    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="01992c2f-0e71-4b86-8e8a-69bb7727bd6a" path="/var/lib/kubelet/pods/01992c2f-0e71-4b86-8e8a-69bb7727bd6a/volumes"
Oct 30 19:43:56.939548 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:56.933175    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="27e1950d-b453-4f32-afd2-95ab9027a2e5" path="/var/lib/kubelet/pods/27e1950d-b453-4f32-afd2-95ab9027a2e5/volumes"
Oct 30 19:43:56.940718 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:56.940689    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="71285665-ae23-4783-a24f-535dccfb68c7" path="/var/lib/kubelet/pods/71285665-ae23-4783-a24f-535dccfb68c7/volumes"
Oct 30 19:43:56.941638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:56.941618    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b6947d3d-3c9a-40d6-a9ae-0604ec1c7125" path="/var/lib/kubelet/pods/b6947d3d-3c9a-40d6-a9ae-0604ec1c7125/volumes"
Oct 30 19:43:56.942264 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:56.942243    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-dd23d1-6968f7db99-q6kfn"]
Oct 30 19:43:56.942476 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:56.942461    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"
Oct 30 19:43:56.942631 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:56.942620    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:43:57.190279 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:57.187937    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4a4e2f-7dc687499d-xwj5f"]
Oct 30 19:43:57.194106 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:43:57.194078    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4a4e2f-5f844996c5-8zh8m"]
Oct 30 19:44:01.090245 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.090182    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w3-dd23d1-dddbddc84-htdcq" podUID="b876f230-c55f-4a4e-bf7d-c6d728ab8be3" containerName="payload-processing" containerID="cri-o://5343c62f8759e651043827b84dff07c6ef3de56bc945d10b3bba8e1ad7486780" gracePeriod=25
Oct 30 19:44:01.272137 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.272097    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-3add95-864947fbf5-59nqp"]
Oct 30 19:44:01.272893 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.272871    5907 cpu_manager.go:414] "containerMap: removing container" podUID="80b91a64-9588-4a51-a88e-c0fc49052ab1" containerName="revoke-keys"
Oct 30 19:44:01.272893 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.272888    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="80b91a64-9588-4a51-a88e-c0fc49052ab1" containerName="revoke-keys"
Oct 30 19:44:01.273016 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.272900    5907 cpu_manager.go:414] "containerMap: removing container" podUID="71285665-ae23-4783-a24f-535dccfb68c7" containerName="istio-proxy"
Oct 30 19:44:01.273016 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.272905    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="71285665-ae23-4783-a24f-535dccfb68c7" containerName="istio-proxy"
Oct 30 19:44:01.273016 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.272919    5907 cpu_manager.go:414] "containerMap: removing container" podUID="b6947d3d-3c9a-40d6-a9ae-0604ec1c7125" containerName="istio-proxy"
Oct 30 19:44:01.273016 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.272924    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b6947d3d-3c9a-40d6-a9ae-0604ec1c7125" containerName="istio-proxy"
Oct 30 19:44:01.273016 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.272935    5907 cpu_manager.go:414] "containerMap: removing container" podUID="01992c2f-0e71-4b86-8e8a-69bb7727bd6a" containerName="istio-proxy"
Oct 30 19:44:01.273016 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.272939    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="01992c2f-0e71-4b86-8e8a-69bb7727bd6a" containerName="istio-proxy"
Oct 30 19:44:01.273016 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.272959    5907 cpu_manager.go:414] "containerMap: removing container" podUID="eb6373d5-f4d9-4110-b9f9-2e1968021aee" containerName="revoke-keys"
Oct 30 19:44:01.273016 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.272964    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eb6373d5-f4d9-4110-b9f9-2e1968021aee" containerName="revoke-keys"
Oct 30 19:44:01.273016 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.272972    5907 cpu_manager.go:414] "containerMap: removing container" podUID="29fbb7be-9068-40ec-a0d4-9fa0ed0469bf" containerName="istio-proxy"
Oct 30 19:44:01.273016 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.272977    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29fbb7be-9068-40ec-a0d4-9fa0ed0469bf" containerName="istio-proxy"
Oct 30 19:44:01.273016 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.272997    5907 cpu_manager.go:414] "containerMap: removing container" podUID="29fbb7be-9068-40ec-a0d4-9fa0ed0469bf" containerName="istio-proxy"
Oct 30 19:44:01.273016 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.273001    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29fbb7be-9068-40ec-a0d4-9fa0ed0469bf" containerName="istio-proxy"
Oct 30 19:44:01.273016 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.273015    5907 cpu_manager.go:414] "containerMap: removing container" podUID="27e1950d-b453-4f32-afd2-95ab9027a2e5" containerName="istio-proxy"
Oct 30 19:44:01.273016 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.273019    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="27e1950d-b453-4f32-afd2-95ab9027a2e5" containerName="istio-proxy"
Oct 30 19:44:01.273465 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.273026    5907 cpu_manager.go:414] "containerMap: removing container" podUID="9e9a9805-721e-4939-a8c4-d078d6a3dc32" containerName="revoke-keys"
Oct 30 19:44:01.273465 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.273031    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9e9a9805-721e-4939-a8c4-d078d6a3dc32" containerName="revoke-keys"
Oct 30 19:44:01.273465 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.273126    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="eb6373d5-f4d9-4110-b9f9-2e1968021aee" containerName="revoke-keys"
Oct 30 19:44:01.273465 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.273136    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="29fbb7be-9068-40ec-a0d4-9fa0ed0469bf" containerName="istio-proxy"
Oct 30 19:44:01.273465 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.273146    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="29fbb7be-9068-40ec-a0d4-9fa0ed0469bf" containerName="istio-proxy"
Oct 30 19:44:01.273465 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.273153    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="27e1950d-b453-4f32-afd2-95ab9027a2e5" containerName="istio-proxy"
Oct 30 19:44:01.273465 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.273158    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="9e9a9805-721e-4939-a8c4-d078d6a3dc32" containerName="revoke-keys"
Oct 30 19:44:01.273465 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.273166    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="80b91a64-9588-4a51-a88e-c0fc49052ab1" containerName="revoke-keys"
Oct 30 19:44:01.273465 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.273173    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="71285665-ae23-4783-a24f-535dccfb68c7" containerName="istio-proxy"
Oct 30 19:44:01.273465 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.273179    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="b6947d3d-3c9a-40d6-a9ae-0604ec1c7125" containerName="istio-proxy"
Oct 30 19:44:01.273465 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.273183    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="01992c2f-0e71-4b86-8e8a-69bb7727bd6a" containerName="istio-proxy"
Oct 30 19:44:01.278790 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.278773    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3add95-864947fbf5-59nqp"
Oct 30 19:44:01.288430 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.288403    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1523289-eee0-4b86-a62f-ee110dde58e6-metrics-certs\") pod \"maas-api-e2e-api-a-3add95-864947fbf5-59nqp\" (UID: \"b1523289-eee0-4b86-a62f-ee110dde58e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3add95-864947fbf5-59nqp"
Oct 30 19:44:01.288548 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.288507    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmnn6\" (UniqueName: \"kubernetes.io/projected/b1523289-eee0-4b86-a62f-ee110dde58e6-kube-api-access-pmnn6\") pod \"maas-api-e2e-api-a-3add95-864947fbf5-59nqp\" (UID: \"b1523289-eee0-4b86-a62f-ee110dde58e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3add95-864947fbf5-59nqp"
Oct 30 19:44:01.288618 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.288556    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1523289-eee0-4b86-a62f-ee110dde58e6-openshift-service-ca\") pod \"maas-api-e2e-api-a-3add95-864947fbf5-59nqp\" (UID: \"b1523289-eee0-4b86-a62f-ee110dde58e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3add95-864947fbf5-59nqp"
Oct 30 19:44:01.288680 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.288625    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1523289-eee0-4b86-a62f-ee110dde58e6-maas-api-tls\") pod \"maas-api-e2e-api-a-3add95-864947fbf5-59nqp\" (UID: \"b1523289-eee0-4b86-a62f-ee110dde58e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3add95-864947fbf5-59nqp"
Oct 30 19:44:01.316623 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.316587    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-3add95-serving-cert\""
Oct 30 19:44:01.319758 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.319728    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-3add95-864947fbf5-59nqp"]
Oct 30 19:44:01.390414 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.390375    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1523289-eee0-4b86-a62f-ee110dde58e6-metrics-certs\") pod \"maas-api-e2e-api-a-3add95-864947fbf5-59nqp\" (UID: \"b1523289-eee0-4b86-a62f-ee110dde58e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3add95-864947fbf5-59nqp"
Oct 30 19:44:01.390600 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.390518    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmnn6\" (UniqueName: \"kubernetes.io/projected/b1523289-eee0-4b86-a62f-ee110dde58e6-kube-api-access-pmnn6\") pod \"maas-api-e2e-api-a-3add95-864947fbf5-59nqp\" (UID: \"b1523289-eee0-4b86-a62f-ee110dde58e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3add95-864947fbf5-59nqp"
Oct 30 19:44:01.390657 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.390599    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1523289-eee0-4b86-a62f-ee110dde58e6-openshift-service-ca\") pod \"maas-api-e2e-api-a-3add95-864947fbf5-59nqp\" (UID: \"b1523289-eee0-4b86-a62f-ee110dde58e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3add95-864947fbf5-59nqp"
Oct 30 19:44:01.390715 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.390665    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1523289-eee0-4b86-a62f-ee110dde58e6-maas-api-tls\") pod \"maas-api-e2e-api-a-3add95-864947fbf5-59nqp\" (UID: \"b1523289-eee0-4b86-a62f-ee110dde58e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3add95-864947fbf5-59nqp"
Oct 30 19:44:01.391369 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.391339    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1523289-eee0-4b86-a62f-ee110dde58e6-openshift-service-ca\") pod \"maas-api-e2e-api-a-3add95-864947fbf5-59nqp\" (UID: \"b1523289-eee0-4b86-a62f-ee110dde58e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3add95-864947fbf5-59nqp"
Oct 30 19:44:01.393225 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.393191    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1523289-eee0-4b86-a62f-ee110dde58e6-metrics-certs\") pod \"maas-api-e2e-api-a-3add95-864947fbf5-59nqp\" (UID: \"b1523289-eee0-4b86-a62f-ee110dde58e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3add95-864947fbf5-59nqp"
Oct 30 19:44:01.393392 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.393375    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1523289-eee0-4b86-a62f-ee110dde58e6-maas-api-tls\") pod \"maas-api-e2e-api-a-3add95-864947fbf5-59nqp\" (UID: \"b1523289-eee0-4b86-a62f-ee110dde58e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3add95-864947fbf5-59nqp"
Oct 30 19:44:01.412494 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.412465    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmnn6\" (UniqueName: \"kubernetes.io/projected/b1523289-eee0-4b86-a62f-ee110dde58e6-kube-api-access-pmnn6\") pod \"maas-api-e2e-api-a-3add95-864947fbf5-59nqp\" (UID: \"b1523289-eee0-4b86-a62f-ee110dde58e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3add95-864947fbf5-59nqp"
Oct 30 19:44:01.597580 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.597523    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3add95-864947fbf5-59nqp"
Oct 30 19:44:01.908905 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.908811    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7263-6d324f"]
Oct 30 19:44:01.914751 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.914727    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7263-6d324f"
Oct 30 19:44:01.929659 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:44:01.929629    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1523289_eee0_4b86_a62f_ee110dde58e6.slice/crio-077e5a61ae8776865614dde5ca3d7dcefe0e816a0af92280992a09b31aabdeb0 WatchSource:0}: Error finding container 077e5a61ae8776865614dde5ca3d7dcefe0e816a0af92280992a09b31aabdeb0: Status 404 returned error can't find the container with id 077e5a61ae8776865614dde5ca3d7dcefe0e816a0af92280992a09b31aabdeb0
Oct 30 19:44:01.930291 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.930267    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-3add95-864947fbf5-59nqp"]
Oct 30 19:44:01.934093 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.934070    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7263-6d324f"]
Oct 30 19:44:01.943686 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.943650    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd23d1-6968f7db99-q6kfn" podUID="b2b39c26-0fce-4ba2-88cb-b171bb43c403" containerName="payload-pre-processing" containerID="cri-o://908c7df9924040b4f2880adea9508d607f52aff4d69393f6376d0251dce243cb" gracePeriod=25
Oct 30 19:44:01.944275 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:01.944254    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-r79h9\""
Oct 30 19:44:02.011546 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:02.011506    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmsv2\" (UniqueName: \"kubernetes.io/projected/375abfba-bce1-4aa5-b7a5-a3cf1527d961-kube-api-access-rmsv2\") pod \"test-curl-7263-6d324f\" (UID: \"375abfba-bce1-4aa5-b7a5-a3cf1527d961\") " pod="opendatahub/test-curl-7263-6d324f"
Oct 30 19:44:02.112608 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:02.112562    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmsv2\" (UniqueName: \"kubernetes.io/projected/375abfba-bce1-4aa5-b7a5-a3cf1527d961-kube-api-access-rmsv2\") pod \"test-curl-7263-6d324f\" (UID: \"375abfba-bce1-4aa5-b7a5-a3cf1527d961\") " pod="opendatahub/test-curl-7263-6d324f"
Oct 30 19:44:02.134529 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:02.134494    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmsv2\" (UniqueName: \"kubernetes.io/projected/375abfba-bce1-4aa5-b7a5-a3cf1527d961-kube-api-access-rmsv2\") pod \"test-curl-7263-6d324f\" (UID: \"375abfba-bce1-4aa5-b7a5-a3cf1527d961\") " pod="opendatahub/test-curl-7263-6d324f"
Oct 30 19:44:02.188506 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:02.188400    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-4a4e2f-7dc687499d-xwj5f" podUID="5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab" containerName="payload-processing" containerID="cri-o://cf13beb2e4b2914a72f2074e59e59f53afa92bd927bd07a55c852230e8b106d2" gracePeriod=25
Oct 30 19:44:02.195006 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:02.194975    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a4e2f-5f844996c5-8zh8m" podUID="347a693d-f331-4300-9bec-d9ee3fafa5a8" containerName="payload-pre-processing" containerID="cri-o://e09a75759f5a2cbfd24fcf135899111c74dab123fb6204094cbaabd4d3516eab" gracePeriod=25
Oct 30 19:44:02.234662 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:02.234632    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7263-6d324f"
Oct 30 19:44:02.692516 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:02.692338    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7263-6d324f"]
Oct 30 19:44:02.723457 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:02.723311    5907 generic.go:356] "Generic (PLEG): container finished" podID="5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab" containerID="cf13beb2e4b2914a72f2074e59e59f53afa92bd927bd07a55c852230e8b106d2" exitCode=0
Oct 30 19:44:02.723457 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:02.723392    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4a4e2f-7dc687499d-xwj5f" event={"ID":"5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab","Type":"ContainerDied","Data":"cf13beb2e4b2914a72f2074e59e59f53afa92bd927bd07a55c852230e8b106d2"}
Oct 30 19:44:02.731093 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:02.730704    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3add95-864947fbf5-59nqp" event={"ID":"b1523289-eee0-4b86-a62f-ee110dde58e6","Type":"ContainerStarted","Data":"077e5a61ae8776865614dde5ca3d7dcefe0e816a0af92280992a09b31aabdeb0"}
Oct 30 19:44:02.734841 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:02.734730    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4a4e2f-7dc687499d-xwj5f"
Oct 30 19:44:02.737173 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:02.737147    5907 generic.go:356] "Generic (PLEG): container finished" podID="347a693d-f331-4300-9bec-d9ee3fafa5a8" containerID="e09a75759f5a2cbfd24fcf135899111c74dab123fb6204094cbaabd4d3516eab" exitCode=0
Oct 30 19:44:02.737493 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:02.737474    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a4e2f-5f844996c5-8zh8m" event={"ID":"347a693d-f331-4300-9bec-d9ee3fafa5a8","Type":"ContainerDied","Data":"e09a75759f5a2cbfd24fcf135899111c74dab123fb6204094cbaabd4d3516eab"}
Oct 30 19:44:02.740285 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:02.740270    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a4e2f-5f844996c5-8zh8m"
Oct 30 19:44:02.918563 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:02.918527    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab-kube-api-access-9xngv\" (UniqueName: \"kubernetes.io/projected/5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab-kube-api-access-9xngv\") pod \"5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab\" (UID: \"5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab\") "
Oct 30 19:44:02.918563 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:02.918563    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/347a693d-f331-4300-9bec-d9ee3fafa5a8-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/347a693d-f331-4300-9bec-d9ee3fafa5a8-plugins-config-volume\") pod \"347a693d-f331-4300-9bec-d9ee3fafa5a8\" (UID: \"347a693d-f331-4300-9bec-d9ee3fafa5a8\") "
Oct 30 19:44:02.918824 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:02.918715    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab-plugins-config-volume\") pod \"5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab\" (UID: \"5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab\") "
Oct 30 19:44:02.918950 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:02.918929    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/347a693d-f331-4300-9bec-d9ee3fafa5a8-kube-api-access-9rh2x\" (UniqueName: \"kubernetes.io/projected/347a693d-f331-4300-9bec-d9ee3fafa5a8-kube-api-access-9rh2x\") pod \"347a693d-f331-4300-9bec-d9ee3fafa5a8\" (UID: \"347a693d-f331-4300-9bec-d9ee3fafa5a8\") "
Oct 30 19:44:02.919063 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:02.919042    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/347a693d-f331-4300-9bec-d9ee3fafa5a8-plugins-config-volume" pod "347a693d-f331-4300-9bec-d9ee3fafa5a8" (UID: "347a693d-f331-4300-9bec-d9ee3fafa5a8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:44:02.919174 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:02.919146    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab-plugins-config-volume" pod "5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab" (UID: "5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:44:02.919522 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:02.919500    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/347a693d-f331-4300-9bec-d9ee3fafa5a8-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:44:02.919522 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:02.919524    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:44:02.921106 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:02.921083    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab-kube-api-access-9xngv" pod "5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab" (UID: "5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab"). InnerVolumeSpecName "kube-api-access-9xngv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:44:02.921195 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:02.921116    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/347a693d-f331-4300-9bec-d9ee3fafa5a8-kube-api-access-9rh2x" pod "347a693d-f331-4300-9bec-d9ee3fafa5a8" (UID: "347a693d-f331-4300-9bec-d9ee3fafa5a8"). InnerVolumeSpecName "kube-api-access-9rh2x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:44:03.021447 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:03.021365    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rh2x\" (UniqueName: \"kubernetes.io/projected/347a693d-f331-4300-9bec-d9ee3fafa5a8-kube-api-access-9rh2x\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:44:03.021447 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:03.021396    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xngv\" (UniqueName: \"kubernetes.io/projected/5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab-kube-api-access-9xngv\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:44:03.750765 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:03.750677    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4a4e2f-7dc687499d-xwj5f"
Oct 30 19:44:03.750765 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:03.750668    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4a4e2f-7dc687499d-xwj5f" event={"ID":"5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab","Type":"ContainerDied","Data":"f4482b4042841d1d68d3dbf8440341af94dbe2d4261bbfc3e8cad0ef486177b1"}
Oct 30 19:44:03.751337 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:03.750793    5907 scope.go:122] "RemoveContainer" containerID="cf13beb2e4b2914a72f2074e59e59f53afa92bd927bd07a55c852230e8b106d2"
Oct 30 19:44:03.764820 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:03.764779    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3add95-864947fbf5-59nqp" event={"ID":"b1523289-eee0-4b86-a62f-ee110dde58e6","Type":"ContainerStarted","Data":"0b76b148be93900aee4a78ab49ba7f5ea5bf61d4fcc1c2526e0415ea81d9b3d2"}
Oct 30 19:44:03.765022 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:03.764922    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3add95-864947fbf5-59nqp"
Oct 30 19:44:03.766835 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:03.766815    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a4e2f-5f844996c5-8zh8m"
Oct 30 19:44:03.767523 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:03.766813    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a4e2f-5f844996c5-8zh8m" event={"ID":"347a693d-f331-4300-9bec-d9ee3fafa5a8","Type":"ContainerDied","Data":"19b3cb611991e16e4b7a7c4e4aa3a400b3d35b51454d34685a1b7c5534d6c148"}
Oct 30 19:44:03.768606 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:03.768574    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7263-6d324f" event={"ID":"375abfba-bce1-4aa5-b7a5-a3cf1527d961","Type":"ContainerStarted","Data":"db060bc3f2097b90dff79b9212f1503ba067b26a99a302d426543a11a8009d1d"}
Oct 30 19:44:03.770087 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:03.770063    5907 scope.go:122] "RemoveContainer" containerID="e09a75759f5a2cbfd24fcf135899111c74dab123fb6204094cbaabd4d3516eab"
Oct 30 19:44:03.811839 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:03.811788    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3add95-864947fbf5-59nqp" podStartSLOduration=2.384336318 podStartE2EDuration="2.811771402s" totalImagesPullingTime="427.435084ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:44:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:44:03.810376991 +0000 UTC m=+1355.653308930" watchObservedRunningTime="2026-10-30 19:44:03.811771402 +0000 UTC m=+1355.654703343"
Oct 30 19:44:03.856809 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:03.856775    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4a4e2f-7dc687499d-xwj5f"]
Oct 30 19:44:03.861770 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:03.861741    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4a4e2f-7dc687499d-xwj5f"]
Oct 30 19:44:03.914565 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:03.914532    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4a4e2f-5f844996c5-8zh8m"]
Oct 30 19:44:03.918716 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:03.918696    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4a4e2f-5f844996c5-8zh8m"]
Oct 30 19:44:04.291543 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:04.291445    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="347a693d-f331-4300-9bec-d9ee3fafa5a8" path="/var/lib/kubelet/pods/347a693d-f331-4300-9bec-d9ee3fafa5a8/volumes"
Oct 30 19:44:04.292128 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:04.292108    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab" path="/var/lib/kubelet/pods/5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab/volumes"
Oct 30 19:44:04.781300 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:04.781265    5907 generic.go:356] "Generic (PLEG): container finished" podID="320e693e-a39d-4e8b-9772-cc8e811acc69" containerID="12edb43d3409e56c27f3fc6e214c9f1184daea65e1a9bdf5f804d720eb82ece0" exitCode=0
Oct 30 19:44:04.781756 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:04.781353    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4" event={"ID":"320e693e-a39d-4e8b-9772-cc8e811acc69","Type":"ContainerDied","Data":"12edb43d3409e56c27f3fc6e214c9f1184daea65e1a9bdf5f804d720eb82ece0"}
Oct 30 19:44:04.781943 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:04.781916    5907 scope.go:122] "RemoveContainer" containerID="12edb43d3409e56c27f3fc6e214c9f1184daea65e1a9bdf5f804d720eb82ece0"
Oct 30 19:44:04.786357 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:04.786322    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:44:04.786496 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:04.786407    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:44:04.786496 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:04.786449    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:44:05.798093 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:05.798005    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7263-6d324f" event={"ID":"375abfba-bce1-4aa5-b7a5-a3cf1527d961","Type":"ContainerStarted","Data":"2e98da10414c6c370db3ca6902b75b80e17370679cd21d8eb4c9d34b891ba72f"}
Oct 30 19:44:05.800409 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:05.800384    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4" event={"ID":"320e693e-a39d-4e8b-9772-cc8e811acc69","Type":"ContainerStarted","Data":"ba19f803954b374f52aded5cc438d5a6c9ebec14088c75e0ef99dc5ab424185c"}
Oct 30 19:44:05.846576 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:05.846517    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7263-6d324f" podStartSLOduration=2.187557886 podStartE2EDuration="4.846497823s" totalImagesPullingTime="2.658939937s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:44:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:44:05.845411009 +0000 UTC m=+1357.688342949" watchObservedRunningTime="2026-10-30 19:44:05.846497823 +0000 UTC m=+1357.689429762"
Oct 30 19:44:06.038710 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:06.038634    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-dd23d1-dddbddc84-htdcq" podUID="b876f230-c55f-4a4e-bf7d-c6d728ab8be3" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:44:06.038962 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:06.038738    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd23d1-6968f7db99-q6kfn" podUID="b2b39c26-0fce-4ba2-88cb-b171bb43c403" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:44:06.460927 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:06.460890    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:44:06.461111 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:06.461055    5907 patch_prober.go:29] interesting pod/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.182:15021/healthz/ready\": dial tcp 10.134.0.182:15021: connect: connection refused" start-of-body=
Oct 30 19:44:06.461111 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:06.461097    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4" podUID="320e693e-a39d-4e8b-9772-cc8e811acc69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.182:15021/healthz/ready\": dial tcp 10.134.0.182:15021: connect: connection refused"
Oct 30 19:44:06.911071 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:06.909887    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7263-6d324f"]
Oct 30 19:44:06.962063 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:06.959239    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7263-6d324f"]
Oct 30 19:44:07.465276 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:07.463977    5907 patch_prober.go:29] interesting pod/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.182:15021/healthz/ready\": dial tcp 10.134.0.182:15021: connect: connection refused" start-of-body=
Oct 30 19:44:07.465276 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:07.464042    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4" podUID="320e693e-a39d-4e8b-9772-cc8e811acc69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.182:15021/healthz/ready\": dial tcp 10.134.0.182:15021: connect: connection refused"
Oct 30 19:44:07.545272 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:07.544746    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-7m79s"]
Oct 30 19:44:07.545754 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:07.545731    5907 cpu_manager.go:414] "containerMap: removing container" podUID="347a693d-f331-4300-9bec-d9ee3fafa5a8" containerName="payload-pre-processing"
Oct 30 19:44:07.545911 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:07.545756    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="347a693d-f331-4300-9bec-d9ee3fafa5a8" containerName="payload-pre-processing"
Oct 30 19:44:07.545911 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:07.545833    5907 cpu_manager.go:414] "containerMap: removing container" podUID="375abfba-bce1-4aa5-b7a5-a3cf1527d961" containerName="test-curl-7263-6d324f"
Oct 30 19:44:07.545911 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:07.545841    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="375abfba-bce1-4aa5-b7a5-a3cf1527d961" containerName="test-curl-7263-6d324f"
Oct 30 19:44:07.545911 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:07.545894    5907 cpu_manager.go:414] "containerMap: removing container" podUID="5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab" containerName="payload-processing"
Oct 30 19:44:07.545911 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:07.545901    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab" containerName="payload-processing"
Oct 30 19:44:07.546179 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:07.546081    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="347a693d-f331-4300-9bec-d9ee3fafa5a8" containerName="payload-pre-processing"
Oct 30 19:44:07.546179 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:07.546111    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="375abfba-bce1-4aa5-b7a5-a3cf1527d961" containerName="test-curl-7263-6d324f"
Oct 30 19:44:07.546179 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:07.546127    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c3f5f70-da3c-4bf0-82fc-1453a7ebd0ab" containerName="payload-processing"
Oct 30 19:44:07.553248 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:07.553220    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-7m79s"
Oct 30 19:44:07.569607 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:07.569577    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-7m79s"]
Oct 30 19:44:07.570545 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:07.570521    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/99a521a7-23c9-43eb-997f-102a3aeb2f62-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-mt-67953efd-7m79s\" (UID: \"99a521a7-23c9-43eb-997f-102a3aeb2f62\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-7m79s"
Oct 30 19:44:07.570671 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:07.570579    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vxxh\" (UniqueName: \"kubernetes.io/projected/99a521a7-23c9-43eb-997f-102a3aeb2f62-kube-api-access-7vxxh\") pod \"maas-api-revoke-keys-e2e-mt-67953efd-7m79s\" (UID: \"99a521a7-23c9-43eb-997f-102a3aeb2f62\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-7m79s"
Oct 30 19:44:07.570745 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:07.570733    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/99a521a7-23c9-43eb-997f-102a3aeb2f62-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-mt-67953efd-7m79s\" (UID: \"99a521a7-23c9-43eb-997f-102a3aeb2f62\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-7m79s"
Oct 30 19:44:07.577101 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:07.577076    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:44:07.577229 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:07.577089    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:44:07.672382 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:07.672343    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vxxh\" (UniqueName: \"kubernetes.io/projected/99a521a7-23c9-43eb-997f-102a3aeb2f62-kube-api-access-7vxxh\") pod \"maas-api-revoke-keys-e2e-mt-67953efd-7m79s\" (UID: \"99a521a7-23c9-43eb-997f-102a3aeb2f62\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-7m79s"
Oct 30 19:44:07.672585 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:07.672517    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/99a521a7-23c9-43eb-997f-102a3aeb2f62-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-mt-67953efd-7m79s\" (UID: \"99a521a7-23c9-43eb-997f-102a3aeb2f62\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-7m79s"
Oct 30 19:44:07.672720 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:07.672702    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/99a521a7-23c9-43eb-997f-102a3aeb2f62-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-mt-67953efd-7m79s\" (UID: \"99a521a7-23c9-43eb-997f-102a3aeb2f62\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-7m79s"
Oct 30 19:44:07.673406 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:07.673317    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/99a521a7-23c9-43eb-997f-102a3aeb2f62-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-mt-67953efd-7m79s\" (UID: \"99a521a7-23c9-43eb-997f-102a3aeb2f62\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-7m79s"
Oct 30 19:44:07.675308 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:07.675283    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/99a521a7-23c9-43eb-997f-102a3aeb2f62-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-mt-67953efd-7m79s\" (UID: \"99a521a7-23c9-43eb-997f-102a3aeb2f62\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-7m79s"
Oct 30 19:44:07.697321 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:07.697297    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vxxh\" (UniqueName: \"kubernetes.io/projected/99a521a7-23c9-43eb-997f-102a3aeb2f62-kube-api-access-7vxxh\") pod \"maas-api-revoke-keys-e2e-mt-67953efd-7m79s\" (UID: \"99a521a7-23c9-43eb-997f-102a3aeb2f62\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-7m79s"
Oct 30 19:44:07.874720 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:07.874680    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-7m79s"
Oct 30 19:44:07.984161 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:07.983494    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7263-6d324f" podUID="375abfba-bce1-4aa5-b7a5-a3cf1527d961" containerName="test-curl-7263-6d324f" containerID="cri-o://2e98da10414c6c370db3ca6902b75b80e17370679cd21d8eb4c9d34b891ba72f" gracePeriod=2
Oct 30 19:44:07.998338 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:44:07.998294    5907 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7263-6d324f\" is forbidden: User \"system:node:ip-10-0-132-152.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-152.ec2.internal' and this object" podUID="375abfba-bce1-4aa5-b7a5-a3cf1527d961" pod="opendatahub/test-curl-7263-6d324f"
Oct 30 19:44:08.239348 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:08.225093    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-7m79s"]
Oct 30 19:44:08.332916 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:44:08.318101    5907 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7263-6d324f\" is forbidden: User \"system:node:ip-10-0-132-152.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-152.ec2.internal' and this object" podUID="375abfba-bce1-4aa5-b7a5-a3cf1527d961" pod="opendatahub/test-curl-7263-6d324f"
Oct 30 19:44:08.460682 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:08.460587    5907 patch_prober.go:29] interesting pod/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.182:15021/healthz/ready\": dial tcp 10.134.0.182:15021: connect: connection refused" start-of-body=
Oct 30 19:44:08.460682 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:08.460650    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4" podUID="320e693e-a39d-4e8b-9772-cc8e811acc69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.182:15021/healthz/ready\": dial tcp 10.134.0.182:15021: connect: connection refused"
Oct 30 19:44:08.994604 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:08.994571    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-7m79s" event={"ID":"99a521a7-23c9-43eb-997f-102a3aeb2f62","Type":"ContainerStarted","Data":"6d668d845dc2e48d3b5c6cac667e97e6d87e5d3a8401fb6bf191d18835db3158"}
Oct 30 19:44:08.995075 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:08.994626    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-7m79s" event={"ID":"99a521a7-23c9-43eb-997f-102a3aeb2f62","Type":"ContainerStarted","Data":"eca8f5461f055278ec2c24700e59f35f3a4ac236fab91b0f59b5cb0cf2e67412"}
Oct 30 19:44:09.054767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:09.054658    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-7m79s" podStartSLOduration=2.054643014 podStartE2EDuration="2.054643014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:44:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:44:09.052137811 +0000 UTC m=+1360.895069745" watchObservedRunningTime="2026-10-30 19:44:09.054643014 +0000 UTC m=+1360.897574952"
Oct 30 19:44:09.463496 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:09.463465    5907 patch_prober.go:29] interesting pod/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.182:15021/healthz/ready\": dial tcp 10.134.0.182:15021: connect: connection refused" start-of-body=
Oct 30 19:44:09.463791 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:09.463763    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4" podUID="320e693e-a39d-4e8b-9772-cc8e811acc69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.182:15021/healthz/ready\": dial tcp 10.134.0.182:15021: connect: connection refused"
Oct 30 19:44:09.790678 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:09.790600    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3add95-864947fbf5-59nqp"
Oct 30 19:44:10.004533 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:10.004505    5907 generic.go:356] "Generic (PLEG): container finished" podID="99a521a7-23c9-43eb-997f-102a3aeb2f62" containerID="6d668d845dc2e48d3b5c6cac667e97e6d87e5d3a8401fb6bf191d18835db3158" exitCode=0
Oct 30 19:44:10.004877 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:10.004720    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-7m79s" event={"ID":"99a521a7-23c9-43eb-997f-102a3aeb2f62","Type":"ContainerDied","Data":"6d668d845dc2e48d3b5c6cac667e97e6d87e5d3a8401fb6bf191d18835db3158"}
Oct 30 19:44:10.281137 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:10.281116    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7263-6d324f"
Oct 30 19:44:10.448178 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:10.448138    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/375abfba-bce1-4aa5-b7a5-a3cf1527d961-kube-api-access-rmsv2\" (UniqueName: \"kubernetes.io/projected/375abfba-bce1-4aa5-b7a5-a3cf1527d961-kube-api-access-rmsv2\") pod \"375abfba-bce1-4aa5-b7a5-a3cf1527d961\" (UID: \"375abfba-bce1-4aa5-b7a5-a3cf1527d961\") "
Oct 30 19:44:10.450550 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:10.450522    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/375abfba-bce1-4aa5-b7a5-a3cf1527d961-kube-api-access-rmsv2" pod "375abfba-bce1-4aa5-b7a5-a3cf1527d961" (UID: "375abfba-bce1-4aa5-b7a5-a3cf1527d961"). InnerVolumeSpecName "kube-api-access-rmsv2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:44:10.460872 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:10.460841    5907 patch_prober.go:29] interesting pod/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.182:15021/healthz/ready\": dial tcp 10.134.0.182:15021: connect: connection refused" start-of-body=
Oct 30 19:44:10.460955 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:10.460925    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4" podUID="320e693e-a39d-4e8b-9772-cc8e811acc69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.182:15021/healthz/ready\": dial tcp 10.134.0.182:15021: connect: connection refused"
Oct 30 19:44:10.549928 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:10.549900    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmsv2\" (UniqueName: \"kubernetes.io/projected/375abfba-bce1-4aa5-b7a5-a3cf1527d961-kube-api-access-rmsv2\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:44:11.065487 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.065447    5907 generic.go:356] "Generic (PLEG): container finished" podID="375abfba-bce1-4aa5-b7a5-a3cf1527d961" containerID="2e98da10414c6c370db3ca6902b75b80e17370679cd21d8eb4c9d34b891ba72f" exitCode=137
Oct 30 19:44:11.066051 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.066021    5907 scope.go:122] "RemoveContainer" containerID="2e98da10414c6c370db3ca6902b75b80e17370679cd21d8eb4c9d34b891ba72f"
Oct 30 19:44:11.066148 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.066054    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7263-6d324f"
Oct 30 19:44:11.081411 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.081387    5907 scope.go:122] "RemoveContainer" containerID="2e98da10414c6c370db3ca6902b75b80e17370679cd21d8eb4c9d34b891ba72f"
Oct 30 19:44:11.081775 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:44:11.081749    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e98da10414c6c370db3ca6902b75b80e17370679cd21d8eb4c9d34b891ba72f\": container with ID starting with 2e98da10414c6c370db3ca6902b75b80e17370679cd21d8eb4c9d34b891ba72f not found: ID does not exist" containerID="2e98da10414c6c370db3ca6902b75b80e17370679cd21d8eb4c9d34b891ba72f"
Oct 30 19:44:11.081835 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.081791    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e98da10414c6c370db3ca6902b75b80e17370679cd21d8eb4c9d34b891ba72f"} err="failed to get container status \"2e98da10414c6c370db3ca6902b75b80e17370679cd21d8eb4c9d34b891ba72f\": rpc error: code = NotFound desc = could not find container \"2e98da10414c6c370db3ca6902b75b80e17370679cd21d8eb4c9d34b891ba72f\": container with ID starting with 2e98da10414c6c370db3ca6902b75b80e17370679cd21d8eb4c9d34b891ba72f not found: ID does not exist"
Oct 30 19:44:11.279300 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.279264    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-7m79s"
Oct 30 19:44:11.369825 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.369792    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/99a521a7-23c9-43eb-997f-102a3aeb2f62-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/99a521a7-23c9-43eb-997f-102a3aeb2f62-maas-api-openshift-ca\") pod \"99a521a7-23c9-43eb-997f-102a3aeb2f62\" (UID: \"99a521a7-23c9-43eb-997f-102a3aeb2f62\") "
Oct 30 19:44:11.370081 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.370060    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/99a521a7-23c9-43eb-997f-102a3aeb2f62-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/99a521a7-23c9-43eb-997f-102a3aeb2f62-maas-api-xks-ca\") pod \"99a521a7-23c9-43eb-997f-102a3aeb2f62\" (UID: \"99a521a7-23c9-43eb-997f-102a3aeb2f62\") "
Oct 30 19:44:11.370187 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.370103    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/99a521a7-23c9-43eb-997f-102a3aeb2f62-kube-api-access-7vxxh\" (UniqueName: \"kubernetes.io/projected/99a521a7-23c9-43eb-997f-102a3aeb2f62-kube-api-access-7vxxh\") pod \"99a521a7-23c9-43eb-997f-102a3aeb2f62\" (UID: \"99a521a7-23c9-43eb-997f-102a3aeb2f62\") "
Oct 30 19:44:11.370251 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.370219    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99a521a7-23c9-43eb-997f-102a3aeb2f62-maas-api-openshift-ca" pod "99a521a7-23c9-43eb-997f-102a3aeb2f62" (UID: "99a521a7-23c9-43eb-997f-102a3aeb2f62"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:44:11.370742 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.370716    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/99a521a7-23c9-43eb-997f-102a3aeb2f62-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:44:11.372604 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.372578    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99a521a7-23c9-43eb-997f-102a3aeb2f62-kube-api-access-7vxxh" pod "99a521a7-23c9-43eb-997f-102a3aeb2f62" (UID: "99a521a7-23c9-43eb-997f-102a3aeb2f62"). InnerVolumeSpecName "kube-api-access-7vxxh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:44:11.372891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.372826    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99a521a7-23c9-43eb-997f-102a3aeb2f62-maas-api-xks-ca" pod "99a521a7-23c9-43eb-997f-102a3aeb2f62" (UID: "99a521a7-23c9-43eb-997f-102a3aeb2f62"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:44:11.472370 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.472337    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/99a521a7-23c9-43eb-997f-102a3aeb2f62-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:44:11.472626 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.472598    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vxxh\" (UniqueName: \"kubernetes.io/projected/99a521a7-23c9-43eb-997f-102a3aeb2f62-kube-api-access-7vxxh\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:44:11.475022 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.475000    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:44:11.475571 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.475543    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:44:11.476795 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.476774    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:44:11.636694 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.636618    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"]
Oct 30 19:44:11.637490 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.637472    5907 cpu_manager.go:414] "containerMap: removing container" podUID="99a521a7-23c9-43eb-997f-102a3aeb2f62" containerName="revoke-keys"
Oct 30 19:44:11.637490 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.637491    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="99a521a7-23c9-43eb-997f-102a3aeb2f62" containerName="revoke-keys"
Oct 30 19:44:11.637674 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.637627    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="99a521a7-23c9-43eb-997f-102a3aeb2f62" containerName="revoke-keys"
Oct 30 19:44:11.641925 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.641905    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:44:11.670071 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.669935    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w0-e0e300-openshift-default-dockercfg-f9ncz\""
Oct 30 19:44:11.676128 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.676088    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57838872-c482-41ec-bda0-4a517fb81dcd-credential-socket\") pod \"e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") " pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:44:11.677165 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.676184    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57838872-c482-41ec-bda0-4a517fb81dcd-istiod-ca-cert\") pod \"e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") " pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:44:11.677165 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.676221    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57838872-c482-41ec-bda0-4a517fb81dcd-workload-certs\") pod \"e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") " pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:44:11.677165 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.676247    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57838872-c482-41ec-bda0-4a517fb81dcd-istio-podinfo\") pod \"e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") " pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:44:11.677165 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.676292    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57838872-c482-41ec-bda0-4a517fb81dcd-istio-token\") pod \"e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") " pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:44:11.677165 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.676390    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57838872-c482-41ec-bda0-4a517fb81dcd-istio-envoy\") pod \"e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") " pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:44:11.677165 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.676510    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57838872-c482-41ec-bda0-4a517fb81dcd-istio-data\") pod \"e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") " pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:44:11.677165 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.676579    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57838872-c482-41ec-bda0-4a517fb81dcd-workload-socket\") pod \"e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") " pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:44:11.677165 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.676649    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9dl4\" (UniqueName: \"kubernetes.io/projected/57838872-c482-41ec-bda0-4a517fb81dcd-kube-api-access-f9dl4\") pod \"e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") " pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:44:11.679075 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.677547    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"]
Oct 30 19:44:11.785882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.785107    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57838872-c482-41ec-bda0-4a517fb81dcd-istio-data\") pod \"e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") " pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:44:11.785882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.785207    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57838872-c482-41ec-bda0-4a517fb81dcd-workload-socket\") pod \"e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") " pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:44:11.785882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.785277    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9dl4\" (UniqueName: \"kubernetes.io/projected/57838872-c482-41ec-bda0-4a517fb81dcd-kube-api-access-f9dl4\") pod \"e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") " pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:44:11.785882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.785321    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57838872-c482-41ec-bda0-4a517fb81dcd-credential-socket\") pod \"e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") " pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:44:11.785882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.785405    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57838872-c482-41ec-bda0-4a517fb81dcd-istiod-ca-cert\") pod \"e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") " pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:44:11.785882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.785438    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57838872-c482-41ec-bda0-4a517fb81dcd-workload-certs\") pod \"e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") " pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:44:11.785882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.785467    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57838872-c482-41ec-bda0-4a517fb81dcd-istio-podinfo\") pod \"e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") " pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:44:11.785882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.785526    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57838872-c482-41ec-bda0-4a517fb81dcd-istio-token\") pod \"e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") " pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:44:11.785882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.785643    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57838872-c482-41ec-bda0-4a517fb81dcd-istio-envoy\") pod \"e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") " pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:44:11.824894 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.819270    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57838872-c482-41ec-bda0-4a517fb81dcd-credential-socket\") pod \"e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") " pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:44:11.824894 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.819543    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57838872-c482-41ec-bda0-4a517fb81dcd-istio-data\") pod \"e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") " pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:44:11.840885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.831504    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57838872-c482-41ec-bda0-4a517fb81dcd-workload-socket\") pod \"e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") " pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:44:11.840885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.832195    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57838872-c482-41ec-bda0-4a517fb81dcd-workload-certs\") pod \"e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") " pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:44:11.840885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.832829    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57838872-c482-41ec-bda0-4a517fb81dcd-istiod-ca-cert\") pod \"e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") " pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:44:11.864888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.859659    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57838872-c482-41ec-bda0-4a517fb81dcd-istio-podinfo\") pod \"e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") " pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:44:11.883996 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.873617    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9dl4\" (UniqueName: \"kubernetes.io/projected/57838872-c482-41ec-bda0-4a517fb81dcd-kube-api-access-f9dl4\") pod \"e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") " pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:44:11.887931 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.885580    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57838872-c482-41ec-bda0-4a517fb81dcd-istio-envoy\") pod \"e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") " pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:44:11.944888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.942583    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57838872-c482-41ec-bda0-4a517fb81dcd-istio-token\") pod \"e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") " pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:44:11.971934 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:11.968663    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:44:12.080545 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:12.080515    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-7m79s"
Oct 30 19:44:12.080545 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:12.080541    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-7m79s" event={"ID":"99a521a7-23c9-43eb-997f-102a3aeb2f62","Type":"ContainerDied","Data":"eca8f5461f055278ec2c24700e59f35f3a4ac236fab91b0f59b5cb0cf2e67412"}
Oct 30 19:44:12.081040 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:12.080594    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eca8f5461f055278ec2c24700e59f35f3a4ac236fab91b0f59b5cb0cf2e67412"
Oct 30 19:44:12.289991 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:12.289910    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="375abfba-bce1-4aa5-b7a5-a3cf1527d961" path="/var/lib/kubelet/pods/375abfba-bce1-4aa5-b7a5-a3cf1527d961/volumes"
Oct 30 19:44:12.390969 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:12.390936    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"]
Oct 30 19:44:12.392298 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:44:12.392275    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57838872_c482_41ec_bda0_4a517fb81dcd.slice/crio-6580af7448b6ee153e9c74962b10eece37c35ade55a9159837fdc0132ccda217 WatchSource:0}: Error finding container 6580af7448b6ee153e9c74962b10eece37c35ade55a9159837fdc0132ccda217: Status 404 returned error can't find the container with id 6580af7448b6ee153e9c74962b10eece37c35ade55a9159837fdc0132ccda217
Oct 30 19:44:12.395155 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:12.395123    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:44:12.395235 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:12.395186    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:44:12.395235 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:12.395216    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:44:13.091552 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:13.091514    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv" event={"ID":"57838872-c482-41ec-bda0-4a517fb81dcd","Type":"ContainerStarted","Data":"1d1433dd49d5c61722711f2b46da6b97cb8aa4b0fc9e164520f3452b073521af"}
Oct 30 19:44:13.092018 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:13.091583    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv" event={"ID":"57838872-c482-41ec-bda0-4a517fb81dcd","Type":"ContainerStarted","Data":"6580af7448b6ee153e9c74962b10eece37c35ade55a9159837fdc0132ccda217"}
Oct 30 19:44:13.161522 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:13.161460    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv" podStartSLOduration=2.161441063 podStartE2EDuration="2.161441063s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:44:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:44:13.157582668 +0000 UTC m=+1365.000514608" watchObservedRunningTime="2026-10-30 19:44:13.161441063 +0000 UTC m=+1365.004373003"
Oct 30 19:44:13.969207 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:13.969175    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:44:13.975680 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:13.975654    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:44:14.099235 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:14.099207    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:44:14.100357 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:14.100335    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:44:16.056139 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:16.056095    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-dd23d1-dddbddc84-htdcq" podUID="b876f230-c55f-4a4e-bf7d-c6d728ab8be3" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:44:16.057301 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:16.057269    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd23d1-6968f7db99-q6kfn" podUID="b2b39c26-0fce-4ba2-88cb-b171bb43c403" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:44:17.503320 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:44:17.503281    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:44:17.692266 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:17.692230    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-e0e300-589877f74c-xnhdg"]
Oct 30 19:44:17.697272 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:17.697248    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-e0e300-589877f74c-xnhdg"
Oct 30 19:44:17.726305 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:17.726275    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w0-e0e300-dockercfg-vtlpf\""
Oct 30 19:44:17.726305 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:17.726401    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w0-e0e300\""
Oct 30 19:44:17.731017 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:17.730990    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-e0e300-589877f74c-xnhdg"]
Oct 30 19:44:17.740922 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:17.736104    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmv2l\" (UniqueName: \"kubernetes.io/projected/63fc1ff9-fc06-4c20-934f-3667809c844e-kube-api-access-wmv2l\") pod \"payload-pre-processing-e2e-shared-b-w0-e0e300-589877f74c-xnhdg\" (UID: \"63fc1ff9-fc06-4c20-934f-3667809c844e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-e0e300-589877f74c-xnhdg"
Oct 30 19:44:17.740922 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:17.736230    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63fc1ff9-fc06-4c20-934f-3667809c844e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w0-e0e300-589877f74c-xnhdg\" (UID: \"63fc1ff9-fc06-4c20-934f-3667809c844e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-e0e300-589877f74c-xnhdg"
Oct 30 19:44:17.910888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:17.909997    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmv2l\" (UniqueName: \"kubernetes.io/projected/63fc1ff9-fc06-4c20-934f-3667809c844e-kube-api-access-wmv2l\") pod \"payload-pre-processing-e2e-shared-b-w0-e0e300-589877f74c-xnhdg\" (UID: \"63fc1ff9-fc06-4c20-934f-3667809c844e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-e0e300-589877f74c-xnhdg"
Oct 30 19:44:17.910888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:17.910158    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63fc1ff9-fc06-4c20-934f-3667809c844e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w0-e0e300-589877f74c-xnhdg\" (UID: \"63fc1ff9-fc06-4c20-934f-3667809c844e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-e0e300-589877f74c-xnhdg"
Oct 30 19:44:17.911197 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:17.911170    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63fc1ff9-fc06-4c20-934f-3667809c844e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w0-e0e300-589877f74c-xnhdg\" (UID: \"63fc1ff9-fc06-4c20-934f-3667809c844e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-e0e300-589877f74c-xnhdg"
Oct 30 19:44:17.961883 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:17.953996    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-e0e300-7b97d947b4-h2v9z"]
Oct 30 19:44:17.989505 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:17.972745    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmv2l\" (UniqueName: \"kubernetes.io/projected/63fc1ff9-fc06-4c20-934f-3667809c844e-kube-api-access-wmv2l\") pod \"payload-pre-processing-e2e-shared-b-w0-e0e300-589877f74c-xnhdg\" (UID: \"63fc1ff9-fc06-4c20-934f-3667809c844e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-e0e300-589877f74c-xnhdg"
Oct 30 19:44:17.989505 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:17.986565    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-e0e300-7b97d947b4-h2v9z"
Oct 30 19:44:18.030202 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:18.030142    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-e0e300-589877f74c-xnhdg"
Oct 30 19:44:18.045534 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:18.045438    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-e0e300-7b97d947b4-h2v9z"]
Oct 30 19:44:18.117455 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:18.117412    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23069174-9b21-412a-aa6c-118d8446da71-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w0-e0e300-7b97d947b4-h2v9z\" (UID: \"23069174-9b21-412a-aa6c-118d8446da71\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-e0e300-7b97d947b4-h2v9z"
Oct 30 19:44:18.123718 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:18.123692    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spgzf\" (UniqueName: \"kubernetes.io/projected/23069174-9b21-412a-aa6c-118d8446da71-kube-api-access-spgzf\") pod \"payload-processing-e2e-shared-b-w0-e0e300-7b97d947b4-h2v9z\" (UID: \"23069174-9b21-412a-aa6c-118d8446da71\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-e0e300-7b97d947b4-h2v9z"
Oct 30 19:44:18.159514 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:18.159478    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-sfh2c"]
Oct 30 19:44:18.225741 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:18.225699    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23069174-9b21-412a-aa6c-118d8446da71-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w0-e0e300-7b97d947b4-h2v9z\" (UID: \"23069174-9b21-412a-aa6c-118d8446da71\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-e0e300-7b97d947b4-h2v9z"
Oct 30 19:44:18.225927 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:18.225834    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spgzf\" (UniqueName: \"kubernetes.io/projected/23069174-9b21-412a-aa6c-118d8446da71-kube-api-access-spgzf\") pod \"payload-processing-e2e-shared-b-w0-e0e300-7b97d947b4-h2v9z\" (UID: \"23069174-9b21-412a-aa6c-118d8446da71\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-e0e300-7b97d947b4-h2v9z"
Oct 30 19:44:18.227830 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:18.227761    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23069174-9b21-412a-aa6c-118d8446da71-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w0-e0e300-7b97d947b4-h2v9z\" (UID: \"23069174-9b21-412a-aa6c-118d8446da71\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-e0e300-7b97d947b4-h2v9z"
Oct 30 19:44:18.255655 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:18.255601    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-spgzf\" (UniqueName: \"kubernetes.io/projected/23069174-9b21-412a-aa6c-118d8446da71-kube-api-access-spgzf\") pod \"payload-processing-e2e-shared-b-w0-e0e300-7b97d947b4-h2v9z\" (UID: \"23069174-9b21-412a-aa6c-118d8446da71\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-e0e300-7b97d947b4-h2v9z"
Oct 30 19:44:18.442941 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:18.427536    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-e0e300-7b97d947b4-h2v9z"
Oct 30 19:44:18.624278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:18.621926    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-e0e300-589877f74c-xnhdg"]
Oct 30 19:44:18.911887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:18.909623    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-e0e300-7b97d947b4-h2v9z"]
Oct 30 19:44:19.146556 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:19.146521    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-e0e300-589877f74c-xnhdg" event={"ID":"63fc1ff9-fc06-4c20-934f-3667809c844e","Type":"ContainerStarted","Data":"438c1a697da47b4710c7bd61c288a453f48bd869bbc92a4fe4e7f2703c1ba4af"}
Oct 30 19:44:19.146726 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:19.146583    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-e0e300-589877f74c-xnhdg" event={"ID":"63fc1ff9-fc06-4c20-934f-3667809c844e","Type":"ContainerStarted","Data":"6f5a095799fff45c12b39f1e10166672229d8e8dd27e1b727bd64b76e436132f"}
Oct 30 19:44:19.146726 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:19.146619    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-e0e300-589877f74c-xnhdg"
Oct 30 19:44:19.153104 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:19.153078    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-e0e300-7b97d947b4-h2v9z" event={"ID":"23069174-9b21-412a-aa6c-118d8446da71","Type":"ContainerStarted","Data":"88adf0683bb6ad8f371b847263c16d4c96fc6a482e8db335f260184a072754c0"}
Oct 30 19:44:19.153236 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:19.153125    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-e0e300-7b97d947b4-h2v9z" event={"ID":"23069174-9b21-412a-aa6c-118d8446da71","Type":"ContainerStarted","Data":"1b95a872787651cc37b8b8dc1008e9ab8c3212e4adc0e49b0540bdb46851fb3d"}
Oct 30 19:44:19.153353 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:19.153329    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-e0e300-7b97d947b4-h2v9z"
Oct 30 19:44:19.196929 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:19.196876    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-e0e300-589877f74c-xnhdg" podStartSLOduration=2.1968424 podStartE2EDuration="2.1968424s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:44:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:44:19.196452188 +0000 UTC m=+1371.039384127" watchObservedRunningTime="2026-10-30 19:44:19.1968424 +0000 UTC m=+1371.039774346"
Oct 30 19:44:19.242085 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:19.241987    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-e0e300-7b97d947b4-h2v9z" podStartSLOduration=2.241974715 podStartE2EDuration="2.241974715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:44:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:44:19.238816564 +0000 UTC m=+1371.081748503" watchObservedRunningTime="2026-10-30 19:44:19.241974715 +0000 UTC m=+1371.084906655"
Oct 30 19:44:21.251239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:21.251206    5907 generic.go:356] "Generic (PLEG): container finished" podID="b876f230-c55f-4a4e-bf7d-c6d728ab8be3" containerID="5343c62f8759e651043827b84dff07c6ef3de56bc945d10b3bba8e1ad7486780" exitCode=0
Oct 30 19:44:21.251616 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:21.251281    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-dd23d1-dddbddc84-htdcq" event={"ID":"b876f230-c55f-4a4e-bf7d-c6d728ab8be3","Type":"ContainerDied","Data":"5343c62f8759e651043827b84dff07c6ef3de56bc945d10b3bba8e1ad7486780"}
Oct 30 19:44:21.384514 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:21.384482    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dbba9287-sx94j"]
Oct 30 19:44:21.389319 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:21.389298    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbba9287-sx94j"
Oct 30 19:44:21.390115 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:21.390092    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-dd23d1-dddbddc84-htdcq"
Oct 30 19:44:21.406259 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:21.406240    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:44:21.420006 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:21.419983    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:44:21.423261 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:21.423237    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dbba9287-sx94j"]
Oct 30 19:44:21.447682 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:21.447650    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b876f230-c55f-4a4e-bf7d-c6d728ab8be3-kube-api-access-frjvc\" (UniqueName: \"kubernetes.io/projected/b876f230-c55f-4a4e-bf7d-c6d728ab8be3-kube-api-access-frjvc\") pod \"b876f230-c55f-4a4e-bf7d-c6d728ab8be3\" (UID: \"b876f230-c55f-4a4e-bf7d-c6d728ab8be3\") "
Oct 30 19:44:21.447831 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:21.447725    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b876f230-c55f-4a4e-bf7d-c6d728ab8be3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b876f230-c55f-4a4e-bf7d-c6d728ab8be3-plugins-config-volume\") pod \"b876f230-c55f-4a4e-bf7d-c6d728ab8be3\" (UID: \"b876f230-c55f-4a4e-bf7d-c6d728ab8be3\") "
Oct 30 19:44:21.448279 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:21.448249    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dc4xv\" (UniqueName: \"kubernetes.io/projected/8a1d7fdf-a22a-4803-91c2-d756ef33f542-kube-api-access-dc4xv\") pod \"maas-api-revoke-sub-dbba9287-sx94j\" (UID: \"8a1d7fdf-a22a-4803-91c2-d756ef33f542\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbba9287-sx94j"
Oct 30 19:44:21.448915 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:21.448889    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b876f230-c55f-4a4e-bf7d-c6d728ab8be3-plugins-config-volume" pod "b876f230-c55f-4a4e-bf7d-c6d728ab8be3" (UID: "b876f230-c55f-4a4e-bf7d-c6d728ab8be3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:44:21.449047 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:21.449027    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8a1d7fdf-a22a-4803-91c2-d756ef33f542-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dbba9287-sx94j\" (UID: \"8a1d7fdf-a22a-4803-91c2-d756ef33f542\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbba9287-sx94j"
Oct 30 19:44:21.449111 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:21.449069    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8a1d7fdf-a22a-4803-91c2-d756ef33f542-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dbba9287-sx94j\" (UID: \"8a1d7fdf-a22a-4803-91c2-d756ef33f542\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbba9287-sx94j"
Oct 30 19:44:21.449242 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:21.449222    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b876f230-c55f-4a4e-bf7d-c6d728ab8be3-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:44:21.451216 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:21.451190    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b876f230-c55f-4a4e-bf7d-c6d728ab8be3-kube-api-access-frjvc" pod "b876f230-c55f-4a4e-bf7d-c6d728ab8be3" (UID: "b876f230-c55f-4a4e-bf7d-c6d728ab8be3"). InnerVolumeSpecName "kube-api-access-frjvc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:44:21.550162 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:21.550125    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8a1d7fdf-a22a-4803-91c2-d756ef33f542-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dbba9287-sx94j\" (UID: \"8a1d7fdf-a22a-4803-91c2-d756ef33f542\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbba9287-sx94j"
Oct 30 19:44:21.550352 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:21.550176    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8a1d7fdf-a22a-4803-91c2-d756ef33f542-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dbba9287-sx94j\" (UID: \"8a1d7fdf-a22a-4803-91c2-d756ef33f542\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbba9287-sx94j"
Oct 30 19:44:21.550352 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:21.550267    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dc4xv\" (UniqueName: \"kubernetes.io/projected/8a1d7fdf-a22a-4803-91c2-d756ef33f542-kube-api-access-dc4xv\") pod \"maas-api-revoke-sub-dbba9287-sx94j\" (UID: \"8a1d7fdf-a22a-4803-91c2-d756ef33f542\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbba9287-sx94j"
Oct 30 19:44:21.550450 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:21.550358    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frjvc\" (UniqueName: \"kubernetes.io/projected/b876f230-c55f-4a4e-bf7d-c6d728ab8be3-kube-api-access-frjvc\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:44:21.550761 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:21.550732    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8a1d7fdf-a22a-4803-91c2-d756ef33f542-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dbba9287-sx94j\" (UID: \"8a1d7fdf-a22a-4803-91c2-d756ef33f542\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbba9287-sx94j"
Oct 30 19:44:21.552742 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:21.552720    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8a1d7fdf-a22a-4803-91c2-d756ef33f542-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dbba9287-sx94j\" (UID: \"8a1d7fdf-a22a-4803-91c2-d756ef33f542\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbba9287-sx94j"
Oct 30 19:44:21.571708 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:21.571676    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dc4xv\" (UniqueName: \"kubernetes.io/projected/8a1d7fdf-a22a-4803-91c2-d756ef33f542-kube-api-access-dc4xv\") pod \"maas-api-revoke-sub-dbba9287-sx94j\" (UID: \"8a1d7fdf-a22a-4803-91c2-d756ef33f542\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbba9287-sx94j"
Oct 30 19:44:21.703573 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:21.703544    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbba9287-sx94j"
Oct 30 19:44:22.037877 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:22.037835    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dbba9287-sx94j"]
Oct 30 19:44:22.038855 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:44:22.038831    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a1d7fdf_a22a_4803_91c2_d756ef33f542.slice/crio-7ab8e30e1da0c74f151dcb3fe845745c73c93ef5aa224bae0d628f498552d4d5 WatchSource:0}: Error finding container 7ab8e30e1da0c74f151dcb3fe845745c73c93ef5aa224bae0d628f498552d4d5: Status 404 returned error can't find the container with id 7ab8e30e1da0c74f151dcb3fe845745c73c93ef5aa224bae0d628f498552d4d5
Oct 30 19:44:22.247241 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:22.247215    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd23d1-6968f7db99-q6kfn"
Oct 30 19:44:22.260415 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:22.260384    5907 generic.go:356] "Generic (PLEG): container finished" podID="b2b39c26-0fce-4ba2-88cb-b171bb43c403" containerID="908c7df9924040b4f2880adea9508d607f52aff4d69393f6376d0251dce243cb" exitCode=0
Oct 30 19:44:22.260764 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:22.260441    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd23d1-6968f7db99-q6kfn"
Oct 30 19:44:22.260764 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:22.260467    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd23d1-6968f7db99-q6kfn" event={"ID":"b2b39c26-0fce-4ba2-88cb-b171bb43c403","Type":"ContainerDied","Data":"908c7df9924040b4f2880adea9508d607f52aff4d69393f6376d0251dce243cb"}
Oct 30 19:44:22.260764 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:22.260537    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-dd23d1-6968f7db99-q6kfn" event={"ID":"b2b39c26-0fce-4ba2-88cb-b171bb43c403","Type":"ContainerDied","Data":"f273d2fd6c5aa39d932d025fb085b83bd52c61f0b5a4953d5f34d2917e36b498"}
Oct 30 19:44:22.260764 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:22.260570    5907 scope.go:122] "RemoveContainer" containerID="908c7df9924040b4f2880adea9508d607f52aff4d69393f6376d0251dce243cb"
Oct 30 19:44:22.261927 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:22.261903    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbba9287-sx94j" event={"ID":"8a1d7fdf-a22a-4803-91c2-d756ef33f542","Type":"ContainerStarted","Data":"7ab8e30e1da0c74f151dcb3fe845745c73c93ef5aa224bae0d628f498552d4d5"}
Oct 30 19:44:22.263588 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:22.263571    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-dd23d1-dddbddc84-htdcq"
Oct 30 19:44:22.263699 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:22.263598    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-dd23d1-dddbddc84-htdcq" event={"ID":"b876f230-c55f-4a4e-bf7d-c6d728ab8be3","Type":"ContainerDied","Data":"84d2b2180a873a1a1502b7dc707a776b572823bb27a947e103002e6d54a9b4ff"}
Oct 30 19:44:22.272373 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:22.272356    5907 scope.go:122] "RemoveContainer" containerID="908c7df9924040b4f2880adea9508d607f52aff4d69393f6376d0251dce243cb"
Oct 30 19:44:22.272629 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:44:22.272607    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"908c7df9924040b4f2880adea9508d607f52aff4d69393f6376d0251dce243cb\": container with ID starting with 908c7df9924040b4f2880adea9508d607f52aff4d69393f6376d0251dce243cb not found: ID does not exist" containerID="908c7df9924040b4f2880adea9508d607f52aff4d69393f6376d0251dce243cb"
Oct 30 19:44:22.272695 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:22.272639    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"908c7df9924040b4f2880adea9508d607f52aff4d69393f6376d0251dce243cb"} err="failed to get container status \"908c7df9924040b4f2880adea9508d607f52aff4d69393f6376d0251dce243cb\": rpc error: code = NotFound desc = could not find container \"908c7df9924040b4f2880adea9508d607f52aff4d69393f6376d0251dce243cb\": container with ID starting with 908c7df9924040b4f2880adea9508d607f52aff4d69393f6376d0251dce243cb not found: ID does not exist"
Oct 30 19:44:22.272695 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:22.272655    5907 scope.go:122] "RemoveContainer" containerID="5343c62f8759e651043827b84dff07c6ef3de56bc945d10b3bba8e1ad7486780"
Oct 30 19:44:22.338226 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:22.338188    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-dd23d1-dddbddc84-htdcq"]
Oct 30 19:44:22.346947 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:22.346920    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-dd23d1-dddbddc84-htdcq"]
Oct 30 19:44:22.387397 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:22.387371    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b2b39c26-0fce-4ba2-88cb-b171bb43c403-kube-api-access-rbjt7\" (UniqueName: \"kubernetes.io/projected/b2b39c26-0fce-4ba2-88cb-b171bb43c403-kube-api-access-rbjt7\") pod \"b2b39c26-0fce-4ba2-88cb-b171bb43c403\" (UID: \"b2b39c26-0fce-4ba2-88cb-b171bb43c403\") "
Oct 30 19:44:22.387526 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:22.387424    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b2b39c26-0fce-4ba2-88cb-b171bb43c403-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2b39c26-0fce-4ba2-88cb-b171bb43c403-plugins-config-volume\") pod \"b2b39c26-0fce-4ba2-88cb-b171bb43c403\" (UID: \"b2b39c26-0fce-4ba2-88cb-b171bb43c403\") "
Oct 30 19:44:22.387852 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:22.387828    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2b39c26-0fce-4ba2-88cb-b171bb43c403-plugins-config-volume" pod "b2b39c26-0fce-4ba2-88cb-b171bb43c403" (UID: "b2b39c26-0fce-4ba2-88cb-b171bb43c403"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:44:22.388180 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:22.388166    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2b39c26-0fce-4ba2-88cb-b171bb43c403-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:44:22.389709 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:22.389690    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2b39c26-0fce-4ba2-88cb-b171bb43c403-kube-api-access-rbjt7" pod "b2b39c26-0fce-4ba2-88cb-b171bb43c403" (UID: "b2b39c26-0fce-4ba2-88cb-b171bb43c403"). InnerVolumeSpecName "kube-api-access-rbjt7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:44:22.489948 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:22.489892    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbjt7\" (UniqueName: \"kubernetes.io/projected/b2b39c26-0fce-4ba2-88cb-b171bb43c403-kube-api-access-rbjt7\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:44:22.612688 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:22.612656    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-dd23d1-6968f7db99-q6kfn"]
Oct 30 19:44:22.624164 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:22.624140    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-dd23d1-6968f7db99-q6kfn"]
Oct 30 19:44:23.160653 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.160599    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-sfh2c" podUID="307b0aff-2386-4219-b34f-5a187231f5a2" containerName="payload-pre-processing" containerID="cri-o://046a92c6ecd48c344653b0d674d38e089b45ad4a4029ef14bc0253d041875c54" gracePeriod=25
Oct 30 19:44:23.179744 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.179716    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-05420cec-nxtf7"]
Oct 30 19:44:23.180545 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.180530    5907 cpu_manager.go:414] "containerMap: removing container" podUID="b2b39c26-0fce-4ba2-88cb-b171bb43c403" containerName="payload-pre-processing"
Oct 30 19:44:23.180587 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.180548    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b2b39c26-0fce-4ba2-88cb-b171bb43c403" containerName="payload-pre-processing"
Oct 30 19:44:23.180587 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.180558    5907 cpu_manager.go:414] "containerMap: removing container" podUID="b876f230-c55f-4a4e-bf7d-c6d728ab8be3" containerName="payload-processing"
Oct 30 19:44:23.180587 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.180564    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b876f230-c55f-4a4e-bf7d-c6d728ab8be3" containerName="payload-processing"
Oct 30 19:44:23.180697 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.180688    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="b2b39c26-0fce-4ba2-88cb-b171bb43c403" containerName="payload-pre-processing"
Oct 30 19:44:23.180736 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.180699    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="b876f230-c55f-4a4e-bf7d-c6d728ab8be3" containerName="payload-processing"
Oct 30 19:44:23.184682 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.184613    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-05420cec-nxtf7"
Oct 30 19:44:23.219987 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.219953    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6c23f404-mm85l"]
Oct 30 19:44:23.225116 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.225092    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-05420cec-nxtf7"]
Oct 30 19:44:23.225282 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.225199    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6c23f404-mm85l"
Oct 30 19:44:23.250355 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.250326    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6c23f404-mm85l"]
Oct 30 19:44:23.281115 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.281072    5907 generic.go:356] "Generic (PLEG): container finished" podID="8a1d7fdf-a22a-4803-91c2-d756ef33f542" containerID="32b031228de61e18f74a60159ca0758df84028d1205b13ca836c34cf937ba214" exitCode=0
Oct 30 19:44:23.281491 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.281130    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbba9287-sx94j" event={"ID":"8a1d7fdf-a22a-4803-91c2-d756ef33f542","Type":"ContainerDied","Data":"32b031228de61e18f74a60159ca0758df84028d1205b13ca836c34cf937ba214"}
Oct 30 19:44:23.302933 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.302887    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0c8a3d58-6f3e-43ef-9adf-910ad6504e87-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-05420cec-nxtf7\" (UID: \"0c8a3d58-6f3e-43ef-9adf-910ad6504e87\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-05420cec-nxtf7"
Oct 30 19:44:23.303161 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.303142    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0c8a3d58-6f3e-43ef-9adf-910ad6504e87-maas-api-xks-ca\") pod \"maas-api-revoke-sub-05420cec-nxtf7\" (UID: \"0c8a3d58-6f3e-43ef-9adf-910ad6504e87\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-05420cec-nxtf7"
Oct 30 19:44:23.303218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.303202    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqxtm\" (UniqueName: \"kubernetes.io/projected/0c8a3d58-6f3e-43ef-9adf-910ad6504e87-kube-api-access-zqxtm\") pod \"maas-api-revoke-sub-05420cec-nxtf7\" (UID: \"0c8a3d58-6f3e-43ef-9adf-910ad6504e87\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-05420cec-nxtf7"
Oct 30 19:44:23.405113 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.405079    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0c8a3d58-6f3e-43ef-9adf-910ad6504e87-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-05420cec-nxtf7\" (UID: \"0c8a3d58-6f3e-43ef-9adf-910ad6504e87\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-05420cec-nxtf7"
Oct 30 19:44:23.405257 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.405175    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/05d43121-612e-4afa-afbd-37776f632f55-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6c23f404-mm85l\" (UID: \"05d43121-612e-4afa-afbd-37776f632f55\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6c23f404-mm85l"
Oct 30 19:44:23.405356 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.405332    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0c8a3d58-6f3e-43ef-9adf-910ad6504e87-maas-api-xks-ca\") pod \"maas-api-revoke-sub-05420cec-nxtf7\" (UID: \"0c8a3d58-6f3e-43ef-9adf-910ad6504e87\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-05420cec-nxtf7"
Oct 30 19:44:23.405420 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.405407    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/05d43121-612e-4afa-afbd-37776f632f55-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6c23f404-mm85l\" (UID: \"05d43121-612e-4afa-afbd-37776f632f55\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6c23f404-mm85l"
Oct 30 19:44:23.405469 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.405442    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqxtm\" (UniqueName: \"kubernetes.io/projected/0c8a3d58-6f3e-43ef-9adf-910ad6504e87-kube-api-access-zqxtm\") pod \"maas-api-revoke-sub-05420cec-nxtf7\" (UID: \"0c8a3d58-6f3e-43ef-9adf-910ad6504e87\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-05420cec-nxtf7"
Oct 30 19:44:23.405527 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.405468    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9gx9\" (UniqueName: \"kubernetes.io/projected/05d43121-612e-4afa-afbd-37776f632f55-kube-api-access-q9gx9\") pod \"maas-api-revoke-sub-6c23f404-mm85l\" (UID: \"05d43121-612e-4afa-afbd-37776f632f55\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6c23f404-mm85l"
Oct 30 19:44:23.405925 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.405902    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0c8a3d58-6f3e-43ef-9adf-910ad6504e87-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-05420cec-nxtf7\" (UID: \"0c8a3d58-6f3e-43ef-9adf-910ad6504e87\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-05420cec-nxtf7"
Oct 30 19:44:23.407933 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.407912    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0c8a3d58-6f3e-43ef-9adf-910ad6504e87-maas-api-xks-ca\") pod \"maas-api-revoke-sub-05420cec-nxtf7\" (UID: \"0c8a3d58-6f3e-43ef-9adf-910ad6504e87\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-05420cec-nxtf7"
Oct 30 19:44:23.426371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.426344    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqxtm\" (UniqueName: \"kubernetes.io/projected/0c8a3d58-6f3e-43ef-9adf-910ad6504e87-kube-api-access-zqxtm\") pod \"maas-api-revoke-sub-05420cec-nxtf7\" (UID: \"0c8a3d58-6f3e-43ef-9adf-910ad6504e87\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-05420cec-nxtf7"
Oct 30 19:44:23.499763 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.499701    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-05420cec-nxtf7"
Oct 30 19:44:23.506643 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.506614    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/05d43121-612e-4afa-afbd-37776f632f55-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6c23f404-mm85l\" (UID: \"05d43121-612e-4afa-afbd-37776f632f55\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6c23f404-mm85l"
Oct 30 19:44:23.506790 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.506771    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/05d43121-612e-4afa-afbd-37776f632f55-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6c23f404-mm85l\" (UID: \"05d43121-612e-4afa-afbd-37776f632f55\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6c23f404-mm85l"
Oct 30 19:44:23.506838 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.506823    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9gx9\" (UniqueName: \"kubernetes.io/projected/05d43121-612e-4afa-afbd-37776f632f55-kube-api-access-q9gx9\") pod \"maas-api-revoke-sub-6c23f404-mm85l\" (UID: \"05d43121-612e-4afa-afbd-37776f632f55\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6c23f404-mm85l"
Oct 30 19:44:23.507314 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.507290    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/05d43121-612e-4afa-afbd-37776f632f55-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6c23f404-mm85l\" (UID: \"05d43121-612e-4afa-afbd-37776f632f55\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6c23f404-mm85l"
Oct 30 19:44:23.509446 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.509426    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/05d43121-612e-4afa-afbd-37776f632f55-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6c23f404-mm85l\" (UID: \"05d43121-612e-4afa-afbd-37776f632f55\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6c23f404-mm85l"
Oct 30 19:44:23.534245 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.534223    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9gx9\" (UniqueName: \"kubernetes.io/projected/05d43121-612e-4afa-afbd-37776f632f55-kube-api-access-q9gx9\") pod \"maas-api-revoke-sub-6c23f404-mm85l\" (UID: \"05d43121-612e-4afa-afbd-37776f632f55\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6c23f404-mm85l"
Oct 30 19:44:23.544206 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.544185    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6c23f404-mm85l"
Oct 30 19:44:23.823342 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.823308    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-05420cec-nxtf7"]
Oct 30 19:44:23.824502 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:44:23.824472    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c8a3d58_6f3e_43ef_9adf_910ad6504e87.slice/crio-031b9fa71fff9345767e85f48dc8839409121533b4b58b2b5bd2d796770b8dae WatchSource:0}: Error finding container 031b9fa71fff9345767e85f48dc8839409121533b4b58b2b5bd2d796770b8dae: Status 404 returned error can't find the container with id 031b9fa71fff9345767e85f48dc8839409121533b4b58b2b5bd2d796770b8dae
Oct 30 19:44:23.980455 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:23.980426    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6c23f404-mm85l"]
Oct 30 19:44:23.981161 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:44:23.981137    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05d43121_612e_4afa_afbd_37776f632f55.slice/crio-10254c4f986a93be75ab0c785e2fc5d1c84aa7d7640c939029067f6702d8ce72 WatchSource:0}: Error finding container 10254c4f986a93be75ab0c785e2fc5d1c84aa7d7640c939029067f6702d8ce72: Status 404 returned error can't find the container with id 10254c4f986a93be75ab0c785e2fc5d1c84aa7d7640c939029067f6702d8ce72
Oct 30 19:44:24.291239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:24.291207    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b2b39c26-0fce-4ba2-88cb-b171bb43c403" path="/var/lib/kubelet/pods/b2b39c26-0fce-4ba2-88cb-b171bb43c403/volumes"
Oct 30 19:44:24.291722 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:24.291705    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b876f230-c55f-4a4e-bf7d-c6d728ab8be3" path="/var/lib/kubelet/pods/b876f230-c55f-4a4e-bf7d-c6d728ab8be3/volumes"
Oct 30 19:44:24.292168 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:24.292140    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-05420cec-nxtf7" event={"ID":"0c8a3d58-6f3e-43ef-9adf-910ad6504e87","Type":"ContainerStarted","Data":"a1d7aa67740480dbcc014018b3d080ea6bfc8d7a76a5fccd42c0dc64f5b9c9cb"}
Oct 30 19:44:24.292308 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:24.292189    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-05420cec-nxtf7" event={"ID":"0c8a3d58-6f3e-43ef-9adf-910ad6504e87","Type":"ContainerStarted","Data":"031b9fa71fff9345767e85f48dc8839409121533b4b58b2b5bd2d796770b8dae"}
Oct 30 19:44:24.293194 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:24.293171    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6c23f404-mm85l" event={"ID":"05d43121-612e-4afa-afbd-37776f632f55","Type":"ContainerStarted","Data":"10254c4f986a93be75ab0c785e2fc5d1c84aa7d7640c939029067f6702d8ce72"}
Oct 30 19:44:24.350763 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:24.350711    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-05420cec-nxtf7" podStartSLOduration=1.350695773 podStartE2EDuration="1.350695773s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:44:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:44:24.348255937 +0000 UTC m=+1376.191187877" watchObservedRunningTime="2026-10-30 19:44:24.350695773 +0000 UTC m=+1376.193627713"
Oct 30 19:44:24.461467 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:24.461433    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbba9287-sx94j"
Oct 30 19:44:24.535137 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:24.535052    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8a1d7fdf-a22a-4803-91c2-d756ef33f542-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8a1d7fdf-a22a-4803-91c2-d756ef33f542-maas-api-openshift-ca\") pod \"8a1d7fdf-a22a-4803-91c2-d756ef33f542\" (UID: \"8a1d7fdf-a22a-4803-91c2-d756ef33f542\") "
Oct 30 19:44:24.535392 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:24.535371    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a1d7fdf-a22a-4803-91c2-d756ef33f542-maas-api-openshift-ca" pod "8a1d7fdf-a22a-4803-91c2-d756ef33f542" (UID: "8a1d7fdf-a22a-4803-91c2-d756ef33f542"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:44:24.535500 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:24.535391    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8a1d7fdf-a22a-4803-91c2-d756ef33f542-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8a1d7fdf-a22a-4803-91c2-d756ef33f542-maas-api-xks-ca\") pod \"8a1d7fdf-a22a-4803-91c2-d756ef33f542\" (UID: \"8a1d7fdf-a22a-4803-91c2-d756ef33f542\") "
Oct 30 19:44:24.535500 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:24.535433    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8a1d7fdf-a22a-4803-91c2-d756ef33f542-kube-api-access-dc4xv\" (UniqueName: \"kubernetes.io/projected/8a1d7fdf-a22a-4803-91c2-d756ef33f542-kube-api-access-dc4xv\") pod \"8a1d7fdf-a22a-4803-91c2-d756ef33f542\" (UID: \"8a1d7fdf-a22a-4803-91c2-d756ef33f542\") "
Oct 30 19:44:24.536170 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:24.536143    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8a1d7fdf-a22a-4803-91c2-d756ef33f542-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:44:24.537903 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:24.537694    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a1d7fdf-a22a-4803-91c2-d756ef33f542-kube-api-access-dc4xv" pod "8a1d7fdf-a22a-4803-91c2-d756ef33f542" (UID: "8a1d7fdf-a22a-4803-91c2-d756ef33f542"). InnerVolumeSpecName "kube-api-access-dc4xv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:44:24.538060 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:24.538042    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a1d7fdf-a22a-4803-91c2-d756ef33f542-maas-api-xks-ca" pod "8a1d7fdf-a22a-4803-91c2-d756ef33f542" (UID: "8a1d7fdf-a22a-4803-91c2-d756ef33f542"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:44:24.637814 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:24.637791    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8a1d7fdf-a22a-4803-91c2-d756ef33f542-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:44:24.637939 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:24.637817    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dc4xv\" (UniqueName: \"kubernetes.io/projected/8a1d7fdf-a22a-4803-91c2-d756ef33f542-kube-api-access-dc4xv\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:44:25.304904 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:25.304852    5907 generic.go:356] "Generic (PLEG): container finished" podID="0c8a3d58-6f3e-43ef-9adf-910ad6504e87" containerID="a1d7aa67740480dbcc014018b3d080ea6bfc8d7a76a5fccd42c0dc64f5b9c9cb" exitCode=0
Oct 30 19:44:25.305319 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:25.304920    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-05420cec-nxtf7" event={"ID":"0c8a3d58-6f3e-43ef-9adf-910ad6504e87","Type":"ContainerDied","Data":"a1d7aa67740480dbcc014018b3d080ea6bfc8d7a76a5fccd42c0dc64f5b9c9cb"}
Oct 30 19:44:25.307843 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:25.307820    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbba9287-sx94j"
Oct 30 19:44:25.308088 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:25.307855    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbba9287-sx94j" event={"ID":"8a1d7fdf-a22a-4803-91c2-d756ef33f542","Type":"ContainerDied","Data":"7ab8e30e1da0c74f151dcb3fe845745c73c93ef5aa224bae0d628f498552d4d5"}
Oct 30 19:44:25.308088 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:25.307933    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7ab8e30e1da0c74f151dcb3fe845745c73c93ef5aa224bae0d628f498552d4d5"
Oct 30 19:44:25.310415 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:25.310388    5907 generic.go:356] "Generic (PLEG): container finished" podID="05d43121-612e-4afa-afbd-37776f632f55" containerID="e17c9ca5a13011ad5df40c26d4ce0883bc8e2587ace8b48f1d3242757209e2e1" exitCode=0
Oct 30 19:44:25.310532 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:25.310464    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6c23f404-mm85l" event={"ID":"05d43121-612e-4afa-afbd-37776f632f55","Type":"ContainerDied","Data":"e17c9ca5a13011ad5df40c26d4ce0883bc8e2587ace8b48f1d3242757209e2e1"}
Oct 30 19:44:26.518591 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:26.518561    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6c23f404-mm85l"
Oct 30 19:44:26.567223 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:26.567197    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-05420cec-nxtf7"
Oct 30 19:44:26.569375 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:26.569351    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/05d43121-612e-4afa-afbd-37776f632f55-kube-api-access-q9gx9\" (UniqueName: \"kubernetes.io/projected/05d43121-612e-4afa-afbd-37776f632f55-kube-api-access-q9gx9\") pod \"05d43121-612e-4afa-afbd-37776f632f55\" (UID: \"05d43121-612e-4afa-afbd-37776f632f55\") "
Oct 30 19:44:26.569492 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:26.569405    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/05d43121-612e-4afa-afbd-37776f632f55-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/05d43121-612e-4afa-afbd-37776f632f55-maas-api-openshift-ca\") pod \"05d43121-612e-4afa-afbd-37776f632f55\" (UID: \"05d43121-612e-4afa-afbd-37776f632f55\") "
Oct 30 19:44:26.569598 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:26.569563    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/05d43121-612e-4afa-afbd-37776f632f55-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/05d43121-612e-4afa-afbd-37776f632f55-maas-api-xks-ca\") pod \"05d43121-612e-4afa-afbd-37776f632f55\" (UID: \"05d43121-612e-4afa-afbd-37776f632f55\") "
Oct 30 19:44:26.569783 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:26.569757    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05d43121-612e-4afa-afbd-37776f632f55-maas-api-openshift-ca" pod "05d43121-612e-4afa-afbd-37776f632f55" (UID: "05d43121-612e-4afa-afbd-37776f632f55"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:44:26.570168 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:26.570148    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/05d43121-612e-4afa-afbd-37776f632f55-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:44:26.572312 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:26.572288    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05d43121-612e-4afa-afbd-37776f632f55-maas-api-xks-ca" pod "05d43121-612e-4afa-afbd-37776f632f55" (UID: "05d43121-612e-4afa-afbd-37776f632f55"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:44:26.576006 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:26.575983    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05d43121-612e-4afa-afbd-37776f632f55-kube-api-access-q9gx9" pod "05d43121-612e-4afa-afbd-37776f632f55" (UID: "05d43121-612e-4afa-afbd-37776f632f55"). InnerVolumeSpecName "kube-api-access-q9gx9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:44:26.671132 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:26.671091    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0c8a3d58-6f3e-43ef-9adf-910ad6504e87-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0c8a3d58-6f3e-43ef-9adf-910ad6504e87-maas-api-openshift-ca\") pod \"0c8a3d58-6f3e-43ef-9adf-910ad6504e87\" (UID: \"0c8a3d58-6f3e-43ef-9adf-910ad6504e87\") "
Oct 30 19:44:26.671614 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:26.671472    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c8a3d58-6f3e-43ef-9adf-910ad6504e87-maas-api-openshift-ca" pod "0c8a3d58-6f3e-43ef-9adf-910ad6504e87" (UID: "0c8a3d58-6f3e-43ef-9adf-910ad6504e87"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:44:26.672022 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:26.671997    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0c8a3d58-6f3e-43ef-9adf-910ad6504e87-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0c8a3d58-6f3e-43ef-9adf-910ad6504e87-maas-api-xks-ca\") pod \"0c8a3d58-6f3e-43ef-9adf-910ad6504e87\" (UID: \"0c8a3d58-6f3e-43ef-9adf-910ad6504e87\") "
Oct 30 19:44:26.672387 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:26.672366    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c8a3d58-6f3e-43ef-9adf-910ad6504e87-kube-api-access-zqxtm\" (UniqueName: \"kubernetes.io/projected/0c8a3d58-6f3e-43ef-9adf-910ad6504e87-kube-api-access-zqxtm\") pod \"0c8a3d58-6f3e-43ef-9adf-910ad6504e87\" (UID: \"0c8a3d58-6f3e-43ef-9adf-910ad6504e87\") "
Oct 30 19:44:26.673199 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:26.673172    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0c8a3d58-6f3e-43ef-9adf-910ad6504e87-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:44:26.673298 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:26.673203    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9gx9\" (UniqueName: \"kubernetes.io/projected/05d43121-612e-4afa-afbd-37776f632f55-kube-api-access-q9gx9\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:44:26.673298 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:26.673219    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/05d43121-612e-4afa-afbd-37776f632f55-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:44:26.675225 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:26.675197    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c8a3d58-6f3e-43ef-9adf-910ad6504e87-maas-api-xks-ca" pod "0c8a3d58-6f3e-43ef-9adf-910ad6504e87" (UID: "0c8a3d58-6f3e-43ef-9adf-910ad6504e87"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:44:26.676231 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:26.676210    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c8a3d58-6f3e-43ef-9adf-910ad6504e87-kube-api-access-zqxtm" pod "0c8a3d58-6f3e-43ef-9adf-910ad6504e87" (UID: "0c8a3d58-6f3e-43ef-9adf-910ad6504e87"). InnerVolumeSpecName "kube-api-access-zqxtm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:44:26.774491 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:26.774459    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0c8a3d58-6f3e-43ef-9adf-910ad6504e87-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:44:26.774491 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:26.774488    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqxtm\" (UniqueName: \"kubernetes.io/projected/0c8a3d58-6f3e-43ef-9adf-910ad6504e87-kube-api-access-zqxtm\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:44:27.329112 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:27.329084    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-05420cec-nxtf7"
Oct 30 19:44:27.329276 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:27.329134    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-05420cec-nxtf7" event={"ID":"0c8a3d58-6f3e-43ef-9adf-910ad6504e87","Type":"ContainerDied","Data":"031b9fa71fff9345767e85f48dc8839409121533b4b58b2b5bd2d796770b8dae"}
Oct 30 19:44:27.329276 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:27.329186    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="031b9fa71fff9345767e85f48dc8839409121533b4b58b2b5bd2d796770b8dae"
Oct 30 19:44:27.330728 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:27.330711    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6c23f404-mm85l"
Oct 30 19:44:27.330876 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:27.330757    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6c23f404-mm85l" event={"ID":"05d43121-612e-4afa-afbd-37776f632f55","Type":"ContainerDied","Data":"10254c4f986a93be75ab0c785e2fc5d1c84aa7d7640c939029067f6702d8ce72"}
Oct 30 19:44:27.330876 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:27.330793    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="10254c4f986a93be75ab0c785e2fc5d1c84aa7d7640c939029067f6702d8ce72"
Oct 30 19:44:29.126630 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:29.126590    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:44:30.245031 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:30.244999    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-e0e300-7b97d947b4-h2v9z"
Oct 30 19:44:30.245637 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:30.245153    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-e0e300-589877f74c-xnhdg"
Oct 30 19:44:30.790805 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:30.790766    5907 scope.go:122] "RemoveContainer" containerID="e8f596f7faf0dcd479eb14d426ee9af3aa93b94b8544f151f06e442474cdb63f"
Oct 30 19:44:30.806646 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:30.806624    5907 scope.go:122] "RemoveContainer" containerID="319f41f4faf68b13cc46cee89239314f8e9e92b66ff427cc50f82d4a8df1be80"
Oct 30 19:44:30.826546 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:30.820065    5907 scope.go:122] "RemoveContainer" containerID="3658afbbef1aa4d00622f94040159a0d7c50079ccd6e553cb9bfce018567fe2f"
Oct 30 19:44:32.605891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:32.604999    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-sfh2c" podUID="307b0aff-2386-4219-b34f-5a187231f5a2" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:44:33.235889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:33.210590    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": context deadline exceeded"
Oct 30 19:44:33.235889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:33.211414    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 19:44:33.303275 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:33.288981    5907 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/health\": context deadline exceeded"
Oct 30 19:44:33.334809 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:33.333561    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:44:33.334809 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:33.334754    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6" podUID="c3f73773-6c2f-47e4-a5a4-c57916f64ac9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.176:8000/ready\": context deadline exceeded"
Oct 30 19:44:33.365584 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:33.365462    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-c2sv6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:44:33.372982 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:33.365553    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c2sv6" podUID="ffa8893a-530b-4232-8e6e-dbe34b30ccda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:44:33.544739 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:33.544635    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:44:33.659543 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:33.659495    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-s826m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:44:33.660243 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:33.660192    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m" podUID="86efaf93-7f28-4e6a-9add-c8bdd87c5261" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:44:34.215213 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:34.213955    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 19:44:34.216610 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:34.215520    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": context deadline exceeded"
Oct 30 19:44:34.237888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:34.217105    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:44:34.238211 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:34.238185    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:44:34.338888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:34.334263    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:44:34.383028 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:34.382981    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6" podUID="c3f73773-6c2f-47e4-a5a4-c57916f64ac9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.176:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:44:34.498088 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:34.497996    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-pdg78 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:44:34.498350 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:34.498322    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-pdg78" podUID="bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:44:34.502694 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:34.502663    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:44:34.503276 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:34.503258    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"
Oct 30 19:44:35.084009 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.083982    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4a4e2f-vp8vp"]
Oct 30 19:44:35.103924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.103888    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4a4e2f-vp8vp"]
Oct 30 19:44:35.104160 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.104000    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:44:35.188891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.185449    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:44:35.232052 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.232022    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"
Oct 30 19:44:35.299936 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.299907    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:44:35.305546 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.305524    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"]
Oct 30 19:44:35.306842 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.306817    5907 cpu_manager.go:414] "containerMap: removing container" podUID="0c8a3d58-6f3e-43ef-9adf-910ad6504e87" containerName="revoke-keys"
Oct 30 19:44:35.307023 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.307009    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0c8a3d58-6f3e-43ef-9adf-910ad6504e87" containerName="revoke-keys"
Oct 30 19:44:35.307367 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.307351    5907 cpu_manager.go:414] "containerMap: removing container" podUID="8a1d7fdf-a22a-4803-91c2-d756ef33f542" containerName="revoke-keys"
Oct 30 19:44:35.307477 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.307464    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8a1d7fdf-a22a-4803-91c2-d756ef33f542" containerName="revoke-keys"
Oct 30 19:44:35.307576 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.307566    5907 cpu_manager.go:414] "containerMap: removing container" podUID="05d43121-612e-4afa-afbd-37776f632f55" containerName="revoke-keys"
Oct 30 19:44:35.307720 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.307709    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="05d43121-612e-4afa-afbd-37776f632f55" containerName="revoke-keys"
Oct 30 19:44:35.308030 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.308013    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="8a1d7fdf-a22a-4803-91c2-d756ef33f542" containerName="revoke-keys"
Oct 30 19:44:35.308149 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.308138    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="05d43121-612e-4afa-afbd-37776f632f55" containerName="revoke-keys"
Oct 30 19:44:35.308242 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.308232    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="0c8a3d58-6f3e-43ef-9adf-910ad6504e87" containerName="revoke-keys"
Oct 30 19:44:35.358208 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.358138    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"]
Oct 30 19:44:35.358484 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.358470    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:44:35.367225 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.367192    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0da36910-7634-4556-b95b-451340c2a584-workload-certs\") pod \"e2e-api-b-9de441-openshift-default-655b8b8797-ps599\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") " pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:44:35.367387 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.367272    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da36910-7634-4556-b95b-451340c2a584-credential-socket\") pod \"e2e-api-b-9de441-openshift-default-655b8b8797-ps599\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") " pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:44:35.367387 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.367315    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0da36910-7634-4556-b95b-451340c2a584-istio-data\") pod \"e2e-api-b-9de441-openshift-default-655b8b8797-ps599\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") " pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:44:35.367387 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.367359    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0da36910-7634-4556-b95b-451340c2a584-istiod-ca-cert\") pod \"e2e-api-b-9de441-openshift-default-655b8b8797-ps599\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") " pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:44:35.367387 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.367381    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0da36910-7634-4556-b95b-451340c2a584-istio-envoy\") pod \"e2e-api-b-9de441-openshift-default-655b8b8797-ps599\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") " pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:44:35.367620 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.367404    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmh6r\" (UniqueName: \"kubernetes.io/projected/0da36910-7634-4556-b95b-451340c2a584-kube-api-access-tmh6r\") pod \"e2e-api-b-9de441-openshift-default-655b8b8797-ps599\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") " pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:44:35.367620 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.367491    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0da36910-7634-4556-b95b-451340c2a584-istio-podinfo\") pod \"e2e-api-b-9de441-openshift-default-655b8b8797-ps599\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") " pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:44:35.367620 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.367541    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0da36910-7634-4556-b95b-451340c2a584-istio-token\") pod \"e2e-api-b-9de441-openshift-default-655b8b8797-ps599\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") " pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:44:35.367813 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.367791    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da36910-7634-4556-b95b-451340c2a584-workload-socket\") pod \"e2e-api-b-9de441-openshift-default-655b8b8797-ps599\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") " pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:44:35.377876 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.377845    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-9de441-openshift-default-dockercfg-jnhtn\""
Oct 30 19:44:35.468994 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.468961    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0da36910-7634-4556-b95b-451340c2a584-workload-certs\") pod \"e2e-api-b-9de441-openshift-default-655b8b8797-ps599\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") " pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:44:35.469182 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.469060    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da36910-7634-4556-b95b-451340c2a584-credential-socket\") pod \"e2e-api-b-9de441-openshift-default-655b8b8797-ps599\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") " pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:44:35.469182 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.469105    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0da36910-7634-4556-b95b-451340c2a584-istio-data\") pod \"e2e-api-b-9de441-openshift-default-655b8b8797-ps599\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") " pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:44:35.469182 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.469168    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0da36910-7634-4556-b95b-451340c2a584-istiod-ca-cert\") pod \"e2e-api-b-9de441-openshift-default-655b8b8797-ps599\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") " pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:44:35.469305 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.469267    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0da36910-7634-4556-b95b-451340c2a584-istio-envoy\") pod \"e2e-api-b-9de441-openshift-default-655b8b8797-ps599\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") " pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:44:35.469362 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.469295    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmh6r\" (UniqueName: \"kubernetes.io/projected/0da36910-7634-4556-b95b-451340c2a584-kube-api-access-tmh6r\") pod \"e2e-api-b-9de441-openshift-default-655b8b8797-ps599\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") " pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:44:35.469436 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.469417    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0da36910-7634-4556-b95b-451340c2a584-istio-podinfo\") pod \"e2e-api-b-9de441-openshift-default-655b8b8797-ps599\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") " pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:44:35.469592 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.469570    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0da36910-7634-4556-b95b-451340c2a584-istio-token\") pod \"e2e-api-b-9de441-openshift-default-655b8b8797-ps599\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") " pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:44:35.469662 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.469646    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da36910-7634-4556-b95b-451340c2a584-workload-socket\") pod \"e2e-api-b-9de441-openshift-default-655b8b8797-ps599\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") " pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:44:35.469732 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.469485    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0da36910-7634-4556-b95b-451340c2a584-workload-certs\") pod \"e2e-api-b-9de441-openshift-default-655b8b8797-ps599\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") " pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:44:35.469909 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.469884    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0da36910-7634-4556-b95b-451340c2a584-istio-data\") pod \"e2e-api-b-9de441-openshift-default-655b8b8797-ps599\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") " pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:44:35.470165 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.469973    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da36910-7634-4556-b95b-451340c2a584-workload-socket\") pod \"e2e-api-b-9de441-openshift-default-655b8b8797-ps599\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") " pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:44:35.470165 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.470067    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0da36910-7634-4556-b95b-451340c2a584-istiod-ca-cert\") pod \"e2e-api-b-9de441-openshift-default-655b8b8797-ps599\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") " pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:44:35.470165 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.470086    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da36910-7634-4556-b95b-451340c2a584-credential-socket\") pod \"e2e-api-b-9de441-openshift-default-655b8b8797-ps599\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") " pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:44:35.471702 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.471677    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0da36910-7634-4556-b95b-451340c2a584-istio-envoy\") pod \"e2e-api-b-9de441-openshift-default-655b8b8797-ps599\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") " pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:44:35.472061 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.472043    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0da36910-7634-4556-b95b-451340c2a584-istio-podinfo\") pod \"e2e-api-b-9de441-openshift-default-655b8b8797-ps599\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") " pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:44:35.497731 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.497704    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0da36910-7634-4556-b95b-451340c2a584-istio-token\") pod \"e2e-api-b-9de441-openshift-default-655b8b8797-ps599\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") " pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:44:35.499610 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.499583    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmh6r\" (UniqueName: \"kubernetes.io/projected/0da36910-7634-4556-b95b-451340c2a584-kube-api-access-tmh6r\") pod \"e2e-api-b-9de441-openshift-default-655b8b8797-ps599\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") " pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:44:35.685893 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:35.685266    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:44:36.239597 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:44:36.239565    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0da36910_7634_4556_b95b_451340c2a584.slice/crio-f3e7ef349e88b49a99f704126acda9526fb27764948df2f8c137a6c7016307b1 WatchSource:0}: Error finding container f3e7ef349e88b49a99f704126acda9526fb27764948df2f8c137a6c7016307b1: Status 404 returned error can't find the container with id f3e7ef349e88b49a99f704126acda9526fb27764948df2f8c137a6c7016307b1
Oct 30 19:44:36.240079 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:36.239994    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"]
Oct 30 19:44:36.243378 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:36.243340    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:44:36.243504 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:36.243425    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:44:36.243504 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:36.243467    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:44:36.294488 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:36.294447    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="80b91a64-9588-4a51-a88e-c0fc49052ab1" path="/var/lib/kubelet/pods/80b91a64-9588-4a51-a88e-c0fc49052ab1/volumes"
Oct 30 19:44:37.259014 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:37.258981    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599" event={"ID":"0da36910-7634-4556-b95b-451340c2a584","Type":"ContainerStarted","Data":"32588ff3a44c0194908475273296014ab5cba4c031e6f156a04a7bda33d45605"}
Oct 30 19:44:37.259434 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:37.259035    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599" event={"ID":"0da36910-7634-4556-b95b-451340c2a584","Type":"ContainerStarted","Data":"f3e7ef349e88b49a99f704126acda9526fb27764948df2f8c137a6c7016307b1"}
Oct 30 19:44:37.340460 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:37.340408    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599" podStartSLOduration=2.340393567 podStartE2EDuration="2.340393567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:44:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:44:37.337633552 +0000 UTC m=+1389.180565491" watchObservedRunningTime="2026-10-30 19:44:37.340393567 +0000 UTC m=+1389.183325573"
Oct 30 19:44:37.685804 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:37.685768    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:44:37.692191 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:37.692164    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:44:38.268937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:38.268904    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:44:38.270209 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:38.270187    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:44:41.690845 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:41.690800    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9de441-67cfb6c4c5-vgswm"]
Oct 30 19:44:41.697090 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:41.697062    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9de441-67cfb6c4c5-vgswm"
Oct 30 19:44:41.723270 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:41.723234    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9de441-67cfb6c4c5-vgswm"]
Oct 30 19:44:41.730994 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:41.730965    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-9de441-dockercfg-5ff5n\""
Oct 30 19:44:41.731167 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:41.731061    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-9de441\""
Oct 30 19:44:41.842886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:41.815234    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a033862c-f1bf-4331-b9dc-ebfa073c5caf-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-9de441-67cfb6c4c5-vgswm\" (UID: \"a033862c-f1bf-4331-b9dc-ebfa073c5caf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9de441-67cfb6c4c5-vgswm"
Oct 30 19:44:41.846110 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:41.846073    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29s4n\" (UniqueName: \"kubernetes.io/projected/a033862c-f1bf-4331-b9dc-ebfa073c5caf-kube-api-access-29s4n\") pod \"payload-pre-processing-e2e-api-b-9de441-67cfb6c4c5-vgswm\" (UID: \"a033862c-f1bf-4331-b9dc-ebfa073c5caf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9de441-67cfb6c4c5-vgswm"
Oct 30 19:44:41.956794 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:41.954312    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a033862c-f1bf-4331-b9dc-ebfa073c5caf-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-9de441-67cfb6c4c5-vgswm\" (UID: \"a033862c-f1bf-4331-b9dc-ebfa073c5caf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9de441-67cfb6c4c5-vgswm"
Oct 30 19:44:41.956794 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:41.954467    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29s4n\" (UniqueName: \"kubernetes.io/projected/a033862c-f1bf-4331-b9dc-ebfa073c5caf-kube-api-access-29s4n\") pod \"payload-pre-processing-e2e-api-b-9de441-67cfb6c4c5-vgswm\" (UID: \"a033862c-f1bf-4331-b9dc-ebfa073c5caf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9de441-67cfb6c4c5-vgswm"
Oct 30 19:44:41.956794 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:41.955693    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a033862c-f1bf-4331-b9dc-ebfa073c5caf-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-9de441-67cfb6c4c5-vgswm\" (UID: \"a033862c-f1bf-4331-b9dc-ebfa073c5caf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9de441-67cfb6c4c5-vgswm"
Oct 30 19:44:41.964263 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:41.963039    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9de441-ffb9d7698-7bgvb"]
Oct 30 19:44:41.974946 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:41.974909    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9de441-ffb9d7698-7bgvb"]
Oct 30 19:44:41.975127 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:41.975069    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-9de441-ffb9d7698-7bgvb"
Oct 30 19:44:42.028709 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:42.020749    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29s4n\" (UniqueName: \"kubernetes.io/projected/a033862c-f1bf-4331-b9dc-ebfa073c5caf-kube-api-access-29s4n\") pod \"payload-pre-processing-e2e-api-b-9de441-67cfb6c4c5-vgswm\" (UID: \"a033862c-f1bf-4331-b9dc-ebfa073c5caf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9de441-67cfb6c4c5-vgswm"
Oct 30 19:44:42.028709 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:42.021062    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9de441-67cfb6c4c5-vgswm"
Oct 30 19:44:42.193557 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:42.192845    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40e15386-d751-4b28-8124-a042609e0fd1-plugins-config-volume\") pod \"payload-processing-e2e-api-b-9de441-ffb9d7698-7bgvb\" (UID: \"40e15386-d751-4b28-8124-a042609e0fd1\") " pod="openshift-ingress/payload-processing-e2e-api-b-9de441-ffb9d7698-7bgvb"
Oct 30 19:44:42.193557 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:42.193003    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drg6m\" (UniqueName: \"kubernetes.io/projected/40e15386-d751-4b28-8124-a042609e0fd1-kube-api-access-drg6m\") pod \"payload-processing-e2e-api-b-9de441-ffb9d7698-7bgvb\" (UID: \"40e15386-d751-4b28-8124-a042609e0fd1\") " pod="openshift-ingress/payload-processing-e2e-api-b-9de441-ffb9d7698-7bgvb"
Oct 30 19:44:42.294614 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:42.294511    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40e15386-d751-4b28-8124-a042609e0fd1-plugins-config-volume\") pod \"payload-processing-e2e-api-b-9de441-ffb9d7698-7bgvb\" (UID: \"40e15386-d751-4b28-8124-a042609e0fd1\") " pod="openshift-ingress/payload-processing-e2e-api-b-9de441-ffb9d7698-7bgvb"
Oct 30 19:44:42.295986 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:42.294639    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drg6m\" (UniqueName: \"kubernetes.io/projected/40e15386-d751-4b28-8124-a042609e0fd1-kube-api-access-drg6m\") pod \"payload-processing-e2e-api-b-9de441-ffb9d7698-7bgvb\" (UID: \"40e15386-d751-4b28-8124-a042609e0fd1\") " pod="openshift-ingress/payload-processing-e2e-api-b-9de441-ffb9d7698-7bgvb"
Oct 30 19:44:42.295986 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:42.295909    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40e15386-d751-4b28-8124-a042609e0fd1-plugins-config-volume\") pod \"payload-processing-e2e-api-b-9de441-ffb9d7698-7bgvb\" (UID: \"40e15386-d751-4b28-8124-a042609e0fd1\") " pod="openshift-ingress/payload-processing-e2e-api-b-9de441-ffb9d7698-7bgvb"
Oct 30 19:44:42.330585 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:42.330561    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drg6m\" (UniqueName: \"kubernetes.io/projected/40e15386-d751-4b28-8124-a042609e0fd1-kube-api-access-drg6m\") pod \"payload-processing-e2e-api-b-9de441-ffb9d7698-7bgvb\" (UID: \"40e15386-d751-4b28-8124-a042609e0fd1\") " pod="openshift-ingress/payload-processing-e2e-api-b-9de441-ffb9d7698-7bgvb"
Oct 30 19:44:42.409575 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:42.409542    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-9de441-ffb9d7698-7bgvb"
Oct 30 19:44:42.426640 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:42.426617    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9de441-67cfb6c4c5-vgswm"]
Oct 30 19:44:42.427057 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:44:42.427035    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda033862c_f1bf_4331_b9dc_ebfa073c5caf.slice/crio-b30898ed4b3451b14d87237697f3fc0a851730b114e08d4e627063ba8688ed96 WatchSource:0}: Error finding container b30898ed4b3451b14d87237697f3fc0a851730b114e08d4e627063ba8688ed96: Status 404 returned error can't find the container with id b30898ed4b3451b14d87237697f3fc0a851730b114e08d4e627063ba8688ed96
Oct 30 19:44:42.586844 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:42.586768    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-sfh2c" podUID="307b0aff-2386-4219-b34f-5a187231f5a2" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:44:42.733128 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:44:42.733090    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40e15386_d751_4b28_8124_a042609e0fd1.slice/crio-f254b0e27b5e0cb9a61314a0d5d7b75c7c2ebfb7e737694fa687980b8909fdec WatchSource:0}: Error finding container f254b0e27b5e0cb9a61314a0d5d7b75c7c2ebfb7e737694fa687980b8909fdec: Status 404 returned error can't find the container with id f254b0e27b5e0cb9a61314a0d5d7b75c7c2ebfb7e737694fa687980b8909fdec
Oct 30 19:44:42.733949 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:42.733922    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9de441-ffb9d7698-7bgvb"]
Oct 30 19:44:43.311553 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:43.311517    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-9de441-ffb9d7698-7bgvb" event={"ID":"40e15386-d751-4b28-8124-a042609e0fd1","Type":"ContainerStarted","Data":"b6631875f83ead306878173b8e367d433836e80d0694971ee92eb1b8d7ca42e1"}
Oct 30 19:44:43.312206 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:43.311591    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-9de441-ffb9d7698-7bgvb" event={"ID":"40e15386-d751-4b28-8124-a042609e0fd1","Type":"ContainerStarted","Data":"f254b0e27b5e0cb9a61314a0d5d7b75c7c2ebfb7e737694fa687980b8909fdec"}
Oct 30 19:44:43.312206 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:43.311753    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-9de441-ffb9d7698-7bgvb"
Oct 30 19:44:43.313537 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:43.313503    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9de441-67cfb6c4c5-vgswm" event={"ID":"a033862c-f1bf-4331-b9dc-ebfa073c5caf","Type":"ContainerStarted","Data":"069ab9d4fac1c81d11a29be7aa25506649cf9a23452815ad58f37dd3869ae8b1"}
Oct 30 19:44:43.313660 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:43.313557    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9de441-67cfb6c4c5-vgswm" event={"ID":"a033862c-f1bf-4331-b9dc-ebfa073c5caf","Type":"ContainerStarted","Data":"b30898ed4b3451b14d87237697f3fc0a851730b114e08d4e627063ba8688ed96"}
Oct 30 19:44:43.313660 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:43.313588    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9de441-67cfb6c4c5-vgswm"
Oct 30 19:44:43.315537 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:43.315510    5907 generic.go:356] "Generic (PLEG): container finished" podID="307b0aff-2386-4219-b34f-5a187231f5a2" containerID="046a92c6ecd48c344653b0d674d38e089b45ad4a4029ef14bc0253d041875c54" exitCode=0
Oct 30 19:44:43.315655 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:43.315576    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-sfh2c" event={"ID":"307b0aff-2386-4219-b34f-5a187231f5a2","Type":"ContainerDied","Data":"046a92c6ecd48c344653b0d674d38e089b45ad4a4029ef14bc0253d041875c54"}
Oct 30 19:44:43.317463 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:43.317436    5907 generic.go:356] "Generic (PLEG): container finished" podID="b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf" containerID="28b30d77ae7ed333981688e96cd6f906eea4c288ff68b8efb0764bbfe4fd7b65" exitCode=0
Oct 30 19:44:43.317574 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:43.317509    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh" event={"ID":"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf","Type":"ContainerDied","Data":"28b30d77ae7ed333981688e96cd6f906eea4c288ff68b8efb0764bbfe4fd7b65"}
Oct 30 19:44:43.317918 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:43.317902    5907 scope.go:122] "RemoveContainer" containerID="28b30d77ae7ed333981688e96cd6f906eea4c288ff68b8efb0764bbfe4fd7b65"
Oct 30 19:44:43.319837 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:43.319801    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:44:43.319962 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:43.319914    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:44:43.320012 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:43.319967    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:44:43.366328 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:43.366270    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-9de441-ffb9d7698-7bgvb" podStartSLOduration=2.366252307 podStartE2EDuration="2.366252307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:44:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:44:43.363100148 +0000 UTC m=+1395.206032111" watchObservedRunningTime="2026-10-30 19:44:43.366252307 +0000 UTC m=+1395.209184247"
Oct 30 19:44:43.418679 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:43.418588    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9de441-67cfb6c4c5-vgswm" podStartSLOduration=2.418574532 podStartE2EDuration="2.418574532s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:44:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:44:43.413350738 +0000 UTC m=+1395.256282670" watchObservedRunningTime="2026-10-30 19:44:43.418574532 +0000 UTC m=+1395.261506472"
Oct 30 19:44:43.454685 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:43.454661    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-sfh2c"
Oct 30 19:44:43.531919 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:43.529099    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/307b0aff-2386-4219-b34f-5a187231f5a2-kube-api-access-mhmhx\" (UniqueName: \"kubernetes.io/projected/307b0aff-2386-4219-b34f-5a187231f5a2-kube-api-access-mhmhx\") pod \"307b0aff-2386-4219-b34f-5a187231f5a2\" (UID: \"307b0aff-2386-4219-b34f-5a187231f5a2\") "
Oct 30 19:44:43.531919 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:43.529153    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/307b0aff-2386-4219-b34f-5a187231f5a2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/307b0aff-2386-4219-b34f-5a187231f5a2-plugins-config-volume\") pod \"307b0aff-2386-4219-b34f-5a187231f5a2\" (UID: \"307b0aff-2386-4219-b34f-5a187231f5a2\") "
Oct 30 19:44:43.531919 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:43.530245    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/307b0aff-2386-4219-b34f-5a187231f5a2-plugins-config-volume" pod "307b0aff-2386-4219-b34f-5a187231f5a2" (UID: "307b0aff-2386-4219-b34f-5a187231f5a2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:44:43.534381 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:43.533414    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/307b0aff-2386-4219-b34f-5a187231f5a2-kube-api-access-mhmhx" pod "307b0aff-2386-4219-b34f-5a187231f5a2" (UID: "307b0aff-2386-4219-b34f-5a187231f5a2"). InnerVolumeSpecName "kube-api-access-mhmhx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:44:43.642912 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:43.638472    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhmhx\" (UniqueName: \"kubernetes.io/projected/307b0aff-2386-4219-b34f-5a187231f5a2-kube-api-access-mhmhx\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:44:43.642912 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:43.638515    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/307b0aff-2386-4219-b34f-5a187231f5a2-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:44:44.342170 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:44.342133    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh" event={"ID":"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf","Type":"ContainerStarted","Data":"5f09ff5c14f414d2bbf80f8daacea879457e1604afa0ab2eb0a3bf37de89d037"}
Oct 30 19:44:44.343993 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:44.343968    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-sfh2c"
Oct 30 19:44:44.344134 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:44.344006    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-sfh2c" event={"ID":"307b0aff-2386-4219-b34f-5a187231f5a2","Type":"ContainerDied","Data":"2d49259e11c6fb6625b4c39893aaa7f64e573690d15a03681cc2601ce4d79560"}
Oct 30 19:44:44.344134 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:44.344070    5907 scope.go:122] "RemoveContainer" containerID="046a92c6ecd48c344653b0d674d38e089b45ad4a4029ef14bc0253d041875c54"
Oct 30 19:44:44.467082 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:44.467044    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-sfh2c"]
Oct 30 19:44:44.471557 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:44.471533    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-sfh2c"]
Oct 30 19:44:44.895953 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:44.895919    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks"]
Oct 30 19:44:44.896728 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:44.896714    5907 cpu_manager.go:414] "containerMap: removing container" podUID="307b0aff-2386-4219-b34f-5a187231f5a2" containerName="payload-pre-processing"
Oct 30 19:44:44.896779 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:44.896730    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="307b0aff-2386-4219-b34f-5a187231f5a2" containerName="payload-pre-processing"
Oct 30 19:44:44.896850 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:44.896841    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="307b0aff-2386-4219-b34f-5a187231f5a2" containerName="payload-pre-processing"
Oct 30 19:44:44.900602 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:44.900584    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks"
Oct 30 19:44:44.921993 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:44.921789    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w0-e0e300-serving-cert\""
Oct 30 19:44:44.944768 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:44.944730    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks"]
Oct 30 19:44:44.962406 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:44.962369    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dde06aaa-256e-46d4-98cf-a762db37475b-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks\" (UID: \"dde06aaa-256e-46d4-98cf-a762db37475b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks"
Oct 30 19:44:44.962589 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:44.962494    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hz5b6\" (UniqueName: \"kubernetes.io/projected/dde06aaa-256e-46d4-98cf-a762db37475b-kube-api-access-hz5b6\") pod \"maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks\" (UID: \"dde06aaa-256e-46d4-98cf-a762db37475b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks"
Oct 30 19:44:44.962701 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:44.962663    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dde06aaa-256e-46d4-98cf-a762db37475b-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks\" (UID: \"dde06aaa-256e-46d4-98cf-a762db37475b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks"
Oct 30 19:44:44.962779 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:44.962746    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dde06aaa-256e-46d4-98cf-a762db37475b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks\" (UID: \"dde06aaa-256e-46d4-98cf-a762db37475b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks"
Oct 30 19:44:45.064572 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:45.064531    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dde06aaa-256e-46d4-98cf-a762db37475b-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks\" (UID: \"dde06aaa-256e-46d4-98cf-a762db37475b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks"
Oct 30 19:44:45.064920 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:45.064622    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dde06aaa-256e-46d4-98cf-a762db37475b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks\" (UID: \"dde06aaa-256e-46d4-98cf-a762db37475b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks"
Oct 30 19:44:45.064920 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:45.064802    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dde06aaa-256e-46d4-98cf-a762db37475b-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks\" (UID: \"dde06aaa-256e-46d4-98cf-a762db37475b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks"
Oct 30 19:44:45.065173 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:45.064918    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hz5b6\" (UniqueName: \"kubernetes.io/projected/dde06aaa-256e-46d4-98cf-a762db37475b-kube-api-access-hz5b6\") pod \"maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks\" (UID: \"dde06aaa-256e-46d4-98cf-a762db37475b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks"
Oct 30 19:44:45.067030 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:45.067005    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dde06aaa-256e-46d4-98cf-a762db37475b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks\" (UID: \"dde06aaa-256e-46d4-98cf-a762db37475b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks"
Oct 30 19:44:45.069731 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:45.069689    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dde06aaa-256e-46d4-98cf-a762db37475b-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks\" (UID: \"dde06aaa-256e-46d4-98cf-a762db37475b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks"
Oct 30 19:44:45.070758 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:45.070733    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dde06aaa-256e-46d4-98cf-a762db37475b-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks\" (UID: \"dde06aaa-256e-46d4-98cf-a762db37475b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks"
Oct 30 19:44:45.091118 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:45.091091    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hz5b6\" (UniqueName: \"kubernetes.io/projected/dde06aaa-256e-46d4-98cf-a762db37475b-kube-api-access-hz5b6\") pod \"maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks\" (UID: \"dde06aaa-256e-46d4-98cf-a762db37475b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks"
Oct 30 19:44:45.103519 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:45.103496    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:44:45.104599 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:45.104574    5907 patch_prober.go:29] interesting pod/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused" start-of-body=
Oct 30 19:44:45.104689 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:45.104631    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh" podUID="b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused"
Oct 30 19:44:45.234016 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:45.231158    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks"
Oct 30 19:44:45.844164 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:45.844134    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks"]
Oct 30 19:44:45.845814 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:44:45.845777    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddde06aaa_256e_46d4_98cf_a762db37475b.slice/crio-73906159cc3aed1874ee5b5e5eddc822012f2095d273f5181eecce856df2bf80 WatchSource:0}: Error finding container 73906159cc3aed1874ee5b5e5eddc822012f2095d273f5181eecce856df2bf80: Status 404 returned error can't find the container with id 73906159cc3aed1874ee5b5e5eddc822012f2095d273f5181eecce856df2bf80
Oct 30 19:44:45.851150 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:45.851126    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-086f2a-flxnc"]
Oct 30 19:44:45.855193 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:45.855178    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-086f2a-flxnc"
Oct 30 19:44:45.875359 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:45.875304    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:44:45.878017 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:45.877998    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-086f2a-flxnc"]
Oct 30 19:44:45.878234 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:45.878214    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:44:45.944568 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:45.944531    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h2vd\" (UniqueName: \"kubernetes.io/projected/d6dc0922-1322-4dfe-b13e-b71af1f9d966-kube-api-access-9h2vd\") pod \"maas-api-revoke-keys-e2e-api-b-086f2a-flxnc\" (UID: \"d6dc0922-1322-4dfe-b13e-b71af1f9d966\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-086f2a-flxnc"
Oct 30 19:44:45.944774 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:45.944606    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d6dc0922-1322-4dfe-b13e-b71af1f9d966-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-api-b-086f2a-flxnc\" (UID: \"d6dc0922-1322-4dfe-b13e-b71af1f9d966\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-086f2a-flxnc"
Oct 30 19:44:45.944928 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:45.944786    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d6dc0922-1322-4dfe-b13e-b71af1f9d966-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-api-b-086f2a-flxnc\" (UID: \"d6dc0922-1322-4dfe-b13e-b71af1f9d966\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-086f2a-flxnc"
Oct 30 19:44:46.052895 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:46.052564    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d6dc0922-1322-4dfe-b13e-b71af1f9d966-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-api-b-086f2a-flxnc\" (UID: \"d6dc0922-1322-4dfe-b13e-b71af1f9d966\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-086f2a-flxnc"
Oct 30 19:44:46.052895 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:46.052805    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h2vd\" (UniqueName: \"kubernetes.io/projected/d6dc0922-1322-4dfe-b13e-b71af1f9d966-kube-api-access-9h2vd\") pod \"maas-api-revoke-keys-e2e-api-b-086f2a-flxnc\" (UID: \"d6dc0922-1322-4dfe-b13e-b71af1f9d966\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-086f2a-flxnc"
Oct 30 19:44:46.053191 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:46.052922    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d6dc0922-1322-4dfe-b13e-b71af1f9d966-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-api-b-086f2a-flxnc\" (UID: \"d6dc0922-1322-4dfe-b13e-b71af1f9d966\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-086f2a-flxnc"
Oct 30 19:44:46.061912 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:46.060534    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d6dc0922-1322-4dfe-b13e-b71af1f9d966-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-api-b-086f2a-flxnc\" (UID: \"d6dc0922-1322-4dfe-b13e-b71af1f9d966\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-086f2a-flxnc"
Oct 30 19:44:46.068988 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:46.068960    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d6dc0922-1322-4dfe-b13e-b71af1f9d966-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-api-b-086f2a-flxnc\" (UID: \"d6dc0922-1322-4dfe-b13e-b71af1f9d966\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-086f2a-flxnc"
Oct 30 19:44:46.094068 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:46.092966    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:44:46.106892 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:46.103022    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h2vd\" (UniqueName: \"kubernetes.io/projected/d6dc0922-1322-4dfe-b13e-b71af1f9d966-kube-api-access-9h2vd\") pod \"maas-api-revoke-keys-e2e-api-b-086f2a-flxnc\" (UID: \"d6dc0922-1322-4dfe-b13e-b71af1f9d966\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-086f2a-flxnc"
Oct 30 19:44:46.107069 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:46.106951    5907 patch_prober.go:29] interesting pod/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused" start-of-body=
Oct 30 19:44:46.107069 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:46.107003    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh" podUID="b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused"
Oct 30 19:44:46.169594 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:46.169467    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-086f2a-flxnc"
Oct 30 19:44:46.297329 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:46.297298    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="307b0aff-2386-4219-b34f-5a187231f5a2" path="/var/lib/kubelet/pods/307b0aff-2386-4219-b34f-5a187231f5a2/volumes"
Oct 30 19:44:46.559982 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:46.559941    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks" event={"ID":"dde06aaa-256e-46d4-98cf-a762db37475b","Type":"ContainerStarted","Data":"46686b1dd6ea0d5d0a901e412801afb72594e73a19c84f2e1c36506e3147ae78"}
Oct 30 19:44:46.560187 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:46.560003    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks" event={"ID":"dde06aaa-256e-46d4-98cf-a762db37475b","Type":"ContainerStarted","Data":"73906159cc3aed1874ee5b5e5eddc822012f2095d273f5181eecce856df2bf80"}
Oct 30 19:44:46.560187 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:46.560050    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks"
Oct 30 19:44:46.569769 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:46.569362    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-086f2a-flxnc"]
Oct 30 19:44:46.572513 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:44:46.572482    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6dc0922_1322_4dfe_b13e_b71af1f9d966.slice/crio-0884b08b9fab79c59ffa530dcf95e472b65928f4b24bcd8314e4093129989e56 WatchSource:0}: Error finding container 0884b08b9fab79c59ffa530dcf95e472b65928f4b24bcd8314e4093129989e56: Status 404 returned error can't find the container with id 0884b08b9fab79c59ffa530dcf95e472b65928f4b24bcd8314e4093129989e56
Oct 30 19:44:46.641339 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:46.641282    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks" podStartSLOduration=2.304381944 podStartE2EDuration="2.641265306s" totalImagesPullingTime="336.883362ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:44:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:44:46.640114701 +0000 UTC m=+1398.483046654" watchObservedRunningTime="2026-10-30 19:44:46.641265306 +0000 UTC m=+1398.484197245"
Oct 30 19:44:47.104842 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:47.104810    5907 patch_prober.go:29] interesting pod/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused" start-of-body=
Oct 30 19:44:47.105303 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:47.104907    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh" podUID="b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused"
Oct 30 19:44:47.589882 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:44:47.589667    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:44:47.605429 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:47.605391    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-086f2a-flxnc" event={"ID":"d6dc0922-1322-4dfe-b13e-b71af1f9d966","Type":"ContainerStarted","Data":"571aa497c82e6d6a784874f21a87b2a1a7fe1d2502b46c0f3fb796790970020a"}
Oct 30 19:44:47.605638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:47.605452    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-086f2a-flxnc" event={"ID":"d6dc0922-1322-4dfe-b13e-b71af1f9d966","Type":"ContainerStarted","Data":"0884b08b9fab79c59ffa530dcf95e472b65928f4b24bcd8314e4093129989e56"}
Oct 30 19:44:47.663102 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:47.663046    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-086f2a-flxnc" podStartSLOduration=2.663030829 podStartE2EDuration="2.663030829s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:44:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:44:47.660160714 +0000 UTC m=+1399.503092653" watchObservedRunningTime="2026-10-30 19:44:47.663030829 +0000 UTC m=+1399.505962768"
Oct 30 19:44:48.104051 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:48.104011    5907 patch_prober.go:29] interesting pod/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused" start-of-body=
Oct 30 19:44:48.104241 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:48.104069    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh" podUID="b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused"
Oct 30 19:44:48.637423 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:48.637390    5907 generic.go:356] "Generic (PLEG): container finished" podID="d6dc0922-1322-4dfe-b13e-b71af1f9d966" containerID="571aa497c82e6d6a784874f21a87b2a1a7fe1d2502b46c0f3fb796790970020a" exitCode=0
Oct 30 19:44:48.638016 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:48.637474    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-086f2a-flxnc" event={"ID":"d6dc0922-1322-4dfe-b13e-b71af1f9d966","Type":"ContainerDied","Data":"571aa497c82e6d6a784874f21a87b2a1a7fe1d2502b46c0f3fb796790970020a"}
Oct 30 19:44:49.104590 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:49.104504    5907 patch_prober.go:29] interesting pod/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused" start-of-body=
Oct 30 19:44:49.104590 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:49.104564    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh" podUID="b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused"
Oct 30 19:44:49.849923 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:49.849896    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-086f2a-flxnc"
Oct 30 19:44:49.954310 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:49.954225    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d6dc0922-1322-4dfe-b13e-b71af1f9d966-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d6dc0922-1322-4dfe-b13e-b71af1f9d966-maas-api-openshift-ca\") pod \"d6dc0922-1322-4dfe-b13e-b71af1f9d966\" (UID: \"d6dc0922-1322-4dfe-b13e-b71af1f9d966\") "
Oct 30 19:44:49.954524 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:49.954502    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d6dc0922-1322-4dfe-b13e-b71af1f9d966-kube-api-access-9h2vd\" (UniqueName: \"kubernetes.io/projected/d6dc0922-1322-4dfe-b13e-b71af1f9d966-kube-api-access-9h2vd\") pod \"d6dc0922-1322-4dfe-b13e-b71af1f9d966\" (UID: \"d6dc0922-1322-4dfe-b13e-b71af1f9d966\") "
Oct 30 19:44:49.954673 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:49.954656    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d6dc0922-1322-4dfe-b13e-b71af1f9d966-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d6dc0922-1322-4dfe-b13e-b71af1f9d966-maas-api-xks-ca\") pod \"d6dc0922-1322-4dfe-b13e-b71af1f9d966\" (UID: \"d6dc0922-1322-4dfe-b13e-b71af1f9d966\") "
Oct 30 19:44:49.954729 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:49.954660    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6dc0922-1322-4dfe-b13e-b71af1f9d966-maas-api-openshift-ca" pod "d6dc0922-1322-4dfe-b13e-b71af1f9d966" (UID: "d6dc0922-1322-4dfe-b13e-b71af1f9d966"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:44:49.965721 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:49.955926    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d6dc0922-1322-4dfe-b13e-b71af1f9d966-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:44:49.966164 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:49.966137    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6dc0922-1322-4dfe-b13e-b71af1f9d966-kube-api-access-9h2vd" pod "d6dc0922-1322-4dfe-b13e-b71af1f9d966" (UID: "d6dc0922-1322-4dfe-b13e-b71af1f9d966"). InnerVolumeSpecName "kube-api-access-9h2vd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:44:49.971783 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:49.971756    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6dc0922-1322-4dfe-b13e-b71af1f9d966-maas-api-xks-ca" pod "d6dc0922-1322-4dfe-b13e-b71af1f9d966" (UID: "d6dc0922-1322-4dfe-b13e-b71af1f9d966"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:44:50.058157 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:50.058125    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9h2vd\" (UniqueName: \"kubernetes.io/projected/d6dc0922-1322-4dfe-b13e-b71af1f9d966-kube-api-access-9h2vd\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:44:50.058157 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:50.058155    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d6dc0922-1322-4dfe-b13e-b71af1f9d966-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:44:50.110055 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:50.110027    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:44:50.111320 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:50.111299    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:44:50.655631 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:50.655602    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-086f2a-flxnc"
Oct 30 19:44:50.655825 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:50.655640    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-086f2a-flxnc" event={"ID":"d6dc0922-1322-4dfe-b13e-b71af1f9d966","Type":"ContainerDied","Data":"0884b08b9fab79c59ffa530dcf95e472b65928f4b24bcd8314e4093129989e56"}
Oct 30 19:44:50.655825 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:50.655690    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0884b08b9fab79c59ffa530dcf95e472b65928f4b24bcd8314e4093129989e56"
Oct 30 19:44:53.643294 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:53.643266    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks"
Oct 30 19:44:54.363190 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:54.363155    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9de441-67cfb6c4c5-vgswm"
Oct 30 19:44:54.370927 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:54.370158    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-9de441-ffb9d7698-7bgvb"
Oct 30 19:44:56.291179 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:56.291149    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-086f2a-9b5f7cf-z2579"]
Oct 30 19:44:56.549033 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:56.548958    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b4530767-v74dj"]
Oct 30 19:44:56.549759 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:56.549744    5907 cpu_manager.go:414] "containerMap: removing container" podUID="d6dc0922-1322-4dfe-b13e-b71af1f9d966" containerName="revoke-keys"
Oct 30 19:44:56.549816 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:56.549760    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d6dc0922-1322-4dfe-b13e-b71af1f9d966" containerName="revoke-keys"
Oct 30 19:44:56.549919 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:56.549909    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="d6dc0922-1322-4dfe-b13e-b71af1f9d966" containerName="revoke-keys"
Oct 30 19:44:56.553916 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:56.553902    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b4530767-v74dj"
Oct 30 19:44:56.576230 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:56.576201    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:44:56.576390 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:56.576342    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:44:56.580828 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:56.580801    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b4530767-v74dj"]
Oct 30 19:44:56.728955 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:56.728919    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c4bf9042-a613-4d6f-8d1d-ad03b6cbc152-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b4530767-v74dj\" (UID: \"c4bf9042-a613-4d6f-8d1d-ad03b6cbc152\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b4530767-v74dj"
Oct 30 19:44:56.729344 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:56.729321    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c4bf9042-a613-4d6f-8d1d-ad03b6cbc152-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b4530767-v74dj\" (UID: \"c4bf9042-a613-4d6f-8d1d-ad03b6cbc152\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b4530767-v74dj"
Oct 30 19:44:56.729551 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:56.729536    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ft4w6\" (UniqueName: \"kubernetes.io/projected/c4bf9042-a613-4d6f-8d1d-ad03b6cbc152-kube-api-access-ft4w6\") pod \"maas-api-revoke-sub-b4530767-v74dj\" (UID: \"c4bf9042-a613-4d6f-8d1d-ad03b6cbc152\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b4530767-v74dj"
Oct 30 19:44:56.831005 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:56.830925    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c4bf9042-a613-4d6f-8d1d-ad03b6cbc152-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b4530767-v74dj\" (UID: \"c4bf9042-a613-4d6f-8d1d-ad03b6cbc152\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b4530767-v74dj"
Oct 30 19:44:56.831169 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:56.831013    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ft4w6\" (UniqueName: \"kubernetes.io/projected/c4bf9042-a613-4d6f-8d1d-ad03b6cbc152-kube-api-access-ft4w6\") pod \"maas-api-revoke-sub-b4530767-v74dj\" (UID: \"c4bf9042-a613-4d6f-8d1d-ad03b6cbc152\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b4530767-v74dj"
Oct 30 19:44:56.831169 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:56.831109    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c4bf9042-a613-4d6f-8d1d-ad03b6cbc152-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b4530767-v74dj\" (UID: \"c4bf9042-a613-4d6f-8d1d-ad03b6cbc152\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b4530767-v74dj"
Oct 30 19:44:56.831579 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:56.831556    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c4bf9042-a613-4d6f-8d1d-ad03b6cbc152-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b4530767-v74dj\" (UID: \"c4bf9042-a613-4d6f-8d1d-ad03b6cbc152\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b4530767-v74dj"
Oct 30 19:44:56.833746 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:56.833718    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c4bf9042-a613-4d6f-8d1d-ad03b6cbc152-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b4530767-v74dj\" (UID: \"c4bf9042-a613-4d6f-8d1d-ad03b6cbc152\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b4530767-v74dj"
Oct 30 19:44:56.852414 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:56.852391    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ft4w6\" (UniqueName: \"kubernetes.io/projected/c4bf9042-a613-4d6f-8d1d-ad03b6cbc152-kube-api-access-ft4w6\") pod \"maas-api-revoke-sub-b4530767-v74dj\" (UID: \"c4bf9042-a613-4d6f-8d1d-ad03b6cbc152\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b4530767-v74dj"
Oct 30 19:44:56.869282 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:56.869257    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b4530767-v74dj"
Oct 30 19:44:57.392613 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:57.392582    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b4530767-v74dj"]
Oct 30 19:44:57.393534 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:44:57.393479    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4bf9042_a613_4d6f_8d1d_ad03b6cbc152.slice/crio-66f6688134b077057f645d9ae1e423f452940d3af06001b8e453118c0a2aa495 WatchSource:0}: Error finding container 66f6688134b077057f645d9ae1e423f452940d3af06001b8e453118c0a2aa495: Status 404 returned error can't find the container with id 66f6688134b077057f645d9ae1e423f452940d3af06001b8e453118c0a2aa495
Oct 30 19:44:57.777886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:57.777056    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b4530767-v74dj" event={"ID":"c4bf9042-a613-4d6f-8d1d-ad03b6cbc152","Type":"ContainerStarted","Data":"66f6688134b077057f645d9ae1e423f452940d3af06001b8e453118c0a2aa495"}
Oct 30 19:44:58.237251 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:58.237158    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-7m79s"]
Oct 30 19:44:58.249100 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:58.249057    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-7m79s"]
Oct 30 19:44:58.329629 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:58.329587    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="99a521a7-23c9-43eb-997f-102a3aeb2f62" path="/var/lib/kubelet/pods/99a521a7-23c9-43eb-997f-102a3aeb2f62/volumes"
Oct 30 19:44:58.787109 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:58.787062    5907 generic.go:356] "Generic (PLEG): container finished" podID="c4bf9042-a613-4d6f-8d1d-ad03b6cbc152" containerID="8630df26dca3f1d1b902c97f9fa09670c86f5763c8d2e92d57b820f5124eb4b2" exitCode=0
Oct 30 19:44:58.787558 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:58.787128    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b4530767-v74dj" event={"ID":"c4bf9042-a613-4d6f-8d1d-ad03b6cbc152","Type":"ContainerDied","Data":"8630df26dca3f1d1b902c97f9fa09670c86f5763c8d2e92d57b820f5124eb4b2"}
Oct 30 19:44:59.983665 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:44:59.983636    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b4530767-v74dj"
Oct 30 19:45:00.145893 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.126537    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c4bf9042-a613-4d6f-8d1d-ad03b6cbc152-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c4bf9042-a613-4d6f-8d1d-ad03b6cbc152-maas-api-openshift-ca\") pod \"c4bf9042-a613-4d6f-8d1d-ad03b6cbc152\" (UID: \"c4bf9042-a613-4d6f-8d1d-ad03b6cbc152\") "
Oct 30 19:45:00.146373 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.146340    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c4bf9042-a613-4d6f-8d1d-ad03b6cbc152-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c4bf9042-a613-4d6f-8d1d-ad03b6cbc152-maas-api-xks-ca\") pod \"c4bf9042-a613-4d6f-8d1d-ad03b6cbc152\" (UID: \"c4bf9042-a613-4d6f-8d1d-ad03b6cbc152\") "
Oct 30 19:45:00.146693 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.146672    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c4bf9042-a613-4d6f-8d1d-ad03b6cbc152-kube-api-access-ft4w6\" (UniqueName: \"kubernetes.io/projected/c4bf9042-a613-4d6f-8d1d-ad03b6cbc152-kube-api-access-ft4w6\") pod \"c4bf9042-a613-4d6f-8d1d-ad03b6cbc152\" (UID: \"c4bf9042-a613-4d6f-8d1d-ad03b6cbc152\") "
Oct 30 19:45:00.150883 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.141880    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4bf9042-a613-4d6f-8d1d-ad03b6cbc152-maas-api-openshift-ca" pod "c4bf9042-a613-4d6f-8d1d-ad03b6cbc152" (UID: "c4bf9042-a613-4d6f-8d1d-ad03b6cbc152"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:00.165885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.151777    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c4bf9042-a613-4d6f-8d1d-ad03b6cbc152-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:00.166138 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.152287    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4bf9042-a613-4d6f-8d1d-ad03b6cbc152-kube-api-access-ft4w6" pod "c4bf9042-a613-4d6f-8d1d-ad03b6cbc152" (UID: "c4bf9042-a613-4d6f-8d1d-ad03b6cbc152"). InnerVolumeSpecName "kube-api-access-ft4w6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:00.179262 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.178812    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4bf9042-a613-4d6f-8d1d-ad03b6cbc152-maas-api-xks-ca" pod "c4bf9042-a613-4d6f-8d1d-ad03b6cbc152" (UID: "c4bf9042-a613-4d6f-8d1d-ad03b6cbc152"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:45:00.279278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.277794    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c4bf9042-a613-4d6f-8d1d-ad03b6cbc152-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:00.279278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.277829    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ft4w6\" (UniqueName: \"kubernetes.io/projected/c4bf9042-a613-4d6f-8d1d-ad03b6cbc152-kube-api-access-ft4w6\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:00.286372 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.285931    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j"]
Oct 30 19:45:00.287711 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.287687    5907 cpu_manager.go:414] "containerMap: removing container" podUID="c4bf9042-a613-4d6f-8d1d-ad03b6cbc152" containerName="revoke-keys"
Oct 30 19:45:00.287939 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.287923    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c4bf9042-a613-4d6f-8d1d-ad03b6cbc152" containerName="revoke-keys"
Oct 30 19:45:00.288276 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.288260    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="c4bf9042-a613-4d6f-8d1d-ad03b6cbc152" containerName="revoke-keys"
Oct 30 19:45:00.301294 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.297718    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j"
Oct 30 19:45:00.408942 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.390373    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vdsg\" (UniqueName: \"kubernetes.io/projected/c8186fe3-60be-4434-aff7-e8251491d6c9-kube-api-access-6vdsg\") pod \"maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j\" (UID: \"c8186fe3-60be-4434-aff7-e8251491d6c9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j"
Oct 30 19:45:00.408942 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.390444    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8186fe3-60be-4434-aff7-e8251491d6c9-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j\" (UID: \"c8186fe3-60be-4434-aff7-e8251491d6c9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j"
Oct 30 19:45:00.501886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.492622    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8186fe3-60be-4434-aff7-e8251491d6c9-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j\" (UID: \"c8186fe3-60be-4434-aff7-e8251491d6c9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j"
Oct 30 19:45:00.533094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.511204    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8186fe3-60be-4434-aff7-e8251491d6c9-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j\" (UID: \"c8186fe3-60be-4434-aff7-e8251491d6c9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j"
Oct 30 19:45:00.580598 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.580542    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vdsg\" (UniqueName: \"kubernetes.io/projected/c8186fe3-60be-4434-aff7-e8251491d6c9-kube-api-access-6vdsg\") pod \"maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j\" (UID: \"c8186fe3-60be-4434-aff7-e8251491d6c9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j"
Oct 30 19:45:00.627234 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.606006    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vdsg\" (UniqueName: \"kubernetes.io/projected/c8186fe3-60be-4434-aff7-e8251491d6c9-kube-api-access-6vdsg\") pod \"maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j\" (UID: \"c8186fe3-60be-4434-aff7-e8251491d6c9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j"
Oct 30 19:45:00.627234 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.608510    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp"]
Oct 30 19:45:00.655654 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.642070    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889825-gkjfd"]
Oct 30 19:45:00.655654 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.648961    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp"
Oct 30 19:45:00.671143 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.670434    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn"]
Oct 30 19:45:00.671691 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.671670    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-gkjfd"
Oct 30 19:45:00.684146 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.684053    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j"]
Oct 30 19:45:00.684146 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.684108    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5"]
Oct 30 19:45:00.685547 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.685523    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn"
Oct 30 19:45:00.707908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.704819    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp"]
Oct 30 19:45:00.707908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.704890    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889825-gkjfd"]
Oct 30 19:45:00.707908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.704923    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn"]
Oct 30 19:45:00.707908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.704960    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9"]
Oct 30 19:45:00.707908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.707178    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5"
Oct 30 19:45:00.717456 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.717428    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw"]
Oct 30 19:45:00.718619 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.718594    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9"
Oct 30 19:45:00.723604 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.723562    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9"]
Oct 30 19:45:00.723705 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.723682    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw"
Oct 30 19:45:00.725555 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.725458    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw"]
Oct 30 19:45:00.733418 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.733385    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5"]
Oct 30 19:45:00.784460 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.784427    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dc2448a-45e5-4f3c-9458-ad45df73bf67-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn\" (UID: \"2dc2448a-45e5-4f3c-9458-ad45df73bf67\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn"
Oct 30 19:45:00.784655 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.784576    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dbb34bb-6526-4dc5-bf66-9850efbdca88-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw\" (UID: \"5dbb34bb-6526-4dc5-bf66-9850efbdca88\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw"
Oct 30 19:45:00.784655 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.784638    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4353f4bc-7a39-4df3-82dd-b6b58ef16b09-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp\" (UID: \"4353f4bc-7a39-4df3-82dd-b6b58ef16b09\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp"
Oct 30 19:45:00.784736 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.784666    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfvf2\" (UniqueName: \"kubernetes.io/projected/2dc2448a-45e5-4f3c-9458-ad45df73bf67-kube-api-access-jfvf2\") pod \"maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn\" (UID: \"2dc2448a-45e5-4f3c-9458-ad45df73bf67\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn"
Oct 30 19:45:00.784823 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.784772    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bba9bd45-e2ea-4652-a1bf-28245075792a-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889825-gkjfd\" (UID: \"bba9bd45-e2ea-4652-a1bf-28245075792a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-gkjfd"
Oct 30 19:45:00.784895 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.784830    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/982777a9-020a-4038-a0a8-c3b0fbef7bc2-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9\" (UID: \"982777a9-020a-4038-a0a8-c3b0fbef7bc2\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9"
Oct 30 19:45:00.784941 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.784901    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25lvf\" (UniqueName: \"kubernetes.io/projected/5dbb34bb-6526-4dc5-bf66-9850efbdca88-kube-api-access-25lvf\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw\" (UID: \"5dbb34bb-6526-4dc5-bf66-9850efbdca88\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw"
Oct 30 19:45:00.784977 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.784958    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fs8wv\" (UniqueName: \"kubernetes.io/projected/a5607fe2-1ded-4e25-8361-baa33c565d00-kube-api-access-fs8wv\") pod \"maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5\" (UID: \"a5607fe2-1ded-4e25-8361-baa33c565d00\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5"
Oct 30 19:45:00.785016 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.784980    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64644\" (UniqueName: \"kubernetes.io/projected/bba9bd45-e2ea-4652-a1bf-28245075792a-kube-api-access-64644\") pod \"maas-api-key-cleanup-29889825-gkjfd\" (UID: \"bba9bd45-e2ea-4652-a1bf-28245075792a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-gkjfd"
Oct 30 19:45:00.785016 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.785000    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5607fe2-1ded-4e25-8361-baa33c565d00-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5\" (UID: \"a5607fe2-1ded-4e25-8361-baa33c565d00\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5"
Oct 30 19:45:00.785221 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.785185    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k868c\" (UniqueName: \"kubernetes.io/projected/982777a9-020a-4038-a0a8-c3b0fbef7bc2-kube-api-access-k868c\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9\" (UID: \"982777a9-020a-4038-a0a8-c3b0fbef7bc2\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9"
Oct 30 19:45:00.785221 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.785211    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5sj6\" (UniqueName: \"kubernetes.io/projected/4353f4bc-7a39-4df3-82dd-b6b58ef16b09-kube-api-access-s5sj6\") pod \"maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp\" (UID: \"4353f4bc-7a39-4df3-82dd-b6b58ef16b09\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp"
Oct 30 19:45:00.804235 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.804203    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b4530767-v74dj"
Oct 30 19:45:00.804401 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.804201    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b4530767-v74dj" event={"ID":"c4bf9042-a613-4d6f-8d1d-ad03b6cbc152","Type":"ContainerDied","Data":"66f6688134b077057f645d9ae1e423f452940d3af06001b8e453118c0a2aa495"}
Oct 30 19:45:00.804401 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.804334    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="66f6688134b077057f645d9ae1e423f452940d3af06001b8e453118c0a2aa495"
Oct 30 19:45:00.835963 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.835933    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j"
Oct 30 19:45:00.887735 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.887700    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dc2448a-45e5-4f3c-9458-ad45df73bf67-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn\" (UID: \"2dc2448a-45e5-4f3c-9458-ad45df73bf67\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn"
Oct 30 19:45:00.887964 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.887788    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dbb34bb-6526-4dc5-bf66-9850efbdca88-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw\" (UID: \"5dbb34bb-6526-4dc5-bf66-9850efbdca88\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw"
Oct 30 19:45:00.887964 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.887815    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4353f4bc-7a39-4df3-82dd-b6b58ef16b09-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp\" (UID: \"4353f4bc-7a39-4df3-82dd-b6b58ef16b09\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp"
Oct 30 19:45:00.887964 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.887833    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfvf2\" (UniqueName: \"kubernetes.io/projected/2dc2448a-45e5-4f3c-9458-ad45df73bf67-kube-api-access-jfvf2\") pod \"maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn\" (UID: \"2dc2448a-45e5-4f3c-9458-ad45df73bf67\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn"
Oct 30 19:45:00.887964 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.887929    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bba9bd45-e2ea-4652-a1bf-28245075792a-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889825-gkjfd\" (UID: \"bba9bd45-e2ea-4652-a1bf-28245075792a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-gkjfd"
Oct 30 19:45:00.888149 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.887991    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/982777a9-020a-4038-a0a8-c3b0fbef7bc2-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9\" (UID: \"982777a9-020a-4038-a0a8-c3b0fbef7bc2\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9"
Oct 30 19:45:00.888149 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.888036    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25lvf\" (UniqueName: \"kubernetes.io/projected/5dbb34bb-6526-4dc5-bf66-9850efbdca88-kube-api-access-25lvf\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw\" (UID: \"5dbb34bb-6526-4dc5-bf66-9850efbdca88\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw"
Oct 30 19:45:00.888149 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.888088    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fs8wv\" (UniqueName: \"kubernetes.io/projected/a5607fe2-1ded-4e25-8361-baa33c565d00-kube-api-access-fs8wv\") pod \"maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5\" (UID: \"a5607fe2-1ded-4e25-8361-baa33c565d00\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5"
Oct 30 19:45:00.888149 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.888121    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64644\" (UniqueName: \"kubernetes.io/projected/bba9bd45-e2ea-4652-a1bf-28245075792a-kube-api-access-64644\") pod \"maas-api-key-cleanup-29889825-gkjfd\" (UID: \"bba9bd45-e2ea-4652-a1bf-28245075792a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-gkjfd"
Oct 30 19:45:00.888304 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.888150    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5607fe2-1ded-4e25-8361-baa33c565d00-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5\" (UID: \"a5607fe2-1ded-4e25-8361-baa33c565d00\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5"
Oct 30 19:45:00.888304 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.888288    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k868c\" (UniqueName: \"kubernetes.io/projected/982777a9-020a-4038-a0a8-c3b0fbef7bc2-kube-api-access-k868c\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9\" (UID: \"982777a9-020a-4038-a0a8-c3b0fbef7bc2\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9"
Oct 30 19:45:00.888388 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.888324    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5sj6\" (UniqueName: \"kubernetes.io/projected/4353f4bc-7a39-4df3-82dd-b6b58ef16b09-kube-api-access-s5sj6\") pod \"maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp\" (UID: \"4353f4bc-7a39-4df3-82dd-b6b58ef16b09\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp"
Oct 30 19:45:00.888527 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.888505    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dc2448a-45e5-4f3c-9458-ad45df73bf67-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn\" (UID: \"2dc2448a-45e5-4f3c-9458-ad45df73bf67\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn"
Oct 30 19:45:00.888579 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.888506    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dbb34bb-6526-4dc5-bf66-9850efbdca88-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw\" (UID: \"5dbb34bb-6526-4dc5-bf66-9850efbdca88\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw"
Oct 30 19:45:00.888664 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.888641    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4353f4bc-7a39-4df3-82dd-b6b58ef16b09-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp\" (UID: \"4353f4bc-7a39-4df3-82dd-b6b58ef16b09\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp"
Oct 30 19:45:00.889144 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.889124    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/982777a9-020a-4038-a0a8-c3b0fbef7bc2-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9\" (UID: \"982777a9-020a-4038-a0a8-c3b0fbef7bc2\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9"
Oct 30 19:45:00.889204 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.889122    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bba9bd45-e2ea-4652-a1bf-28245075792a-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889825-gkjfd\" (UID: \"bba9bd45-e2ea-4652-a1bf-28245075792a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-gkjfd"
Oct 30 19:45:00.889298 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.889281    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5607fe2-1ded-4e25-8361-baa33c565d00-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5\" (UID: \"a5607fe2-1ded-4e25-8361-baa33c565d00\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5"
Oct 30 19:45:00.927669 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.927023    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k868c\" (UniqueName: \"kubernetes.io/projected/982777a9-020a-4038-a0a8-c3b0fbef7bc2-kube-api-access-k868c\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9\" (UID: \"982777a9-020a-4038-a0a8-c3b0fbef7bc2\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9"
Oct 30 19:45:00.935451 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.934812    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fs8wv\" (UniqueName: \"kubernetes.io/projected/a5607fe2-1ded-4e25-8361-baa33c565d00-kube-api-access-fs8wv\") pod \"maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5\" (UID: \"a5607fe2-1ded-4e25-8361-baa33c565d00\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5"
Oct 30 19:45:00.935451 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.935314    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfvf2\" (UniqueName: \"kubernetes.io/projected/2dc2448a-45e5-4f3c-9458-ad45df73bf67-kube-api-access-jfvf2\") pod \"maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn\" (UID: \"2dc2448a-45e5-4f3c-9458-ad45df73bf67\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn"
Oct 30 19:45:00.935451 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.935365    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64644\" (UniqueName: \"kubernetes.io/projected/bba9bd45-e2ea-4652-a1bf-28245075792a-kube-api-access-64644\") pod \"maas-api-key-cleanup-29889825-gkjfd\" (UID: \"bba9bd45-e2ea-4652-a1bf-28245075792a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-gkjfd"
Oct 30 19:45:00.935451 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.935383    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5sj6\" (UniqueName: \"kubernetes.io/projected/4353f4bc-7a39-4df3-82dd-b6b58ef16b09-kube-api-access-s5sj6\") pod \"maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp\" (UID: \"4353f4bc-7a39-4df3-82dd-b6b58ef16b09\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp"
Oct 30 19:45:00.945241 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.945214    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25lvf\" (UniqueName: \"kubernetes.io/projected/5dbb34bb-6526-4dc5-bf66-9850efbdca88-kube-api-access-25lvf\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw\" (UID: \"5dbb34bb-6526-4dc5-bf66-9850efbdca88\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw"
Oct 30 19:45:00.976817 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:00.976773    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp"
Oct 30 19:45:01.001808 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:01.001777    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-gkjfd"
Oct 30 19:45:01.031344 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:01.031309    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn"
Oct 30 19:45:01.041733 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:01.041301    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5"
Oct 30 19:45:01.054041 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:01.049200    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9"
Oct 30 19:45:01.056539 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:01.056514    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw"
Oct 30 19:45:01.298725 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:01.296620    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-086f2a-9b5f7cf-z2579" podUID="d925ef89-ce40-4ed1-ad1a-eadd4778245e" containerName="payload-pre-processing" containerID="cri-o://6fd5df0e49899c37be4b643fdbc1e795b2606432a7252a19ba46e5eac073385f" gracePeriod=25
Oct 30 19:45:01.343910 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:01.343571    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-dd23d1-mngxw"]
Oct 30 19:45:01.352533 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:01.351963    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-dd23d1-mngxw"]
Oct 30 19:45:01.360754 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:01.357813    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j"]
Oct 30 19:45:01.517062 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:01.517026    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp"]
Oct 30 19:45:01.538072 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:01.538043    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9"]
Oct 30 19:45:01.540447 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:45:01.540419    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod982777a9_020a_4038_a0a8_c3b0fbef7bc2.slice/crio-1b4d5332c1bfd9b86a98053e48e403570120d592dd09c60f5b4af625161c17c0 WatchSource:0}: Error finding container 1b4d5332c1bfd9b86a98053e48e403570120d592dd09c60f5b4af625161c17c0: Status 404 returned error can't find the container with id 1b4d5332c1bfd9b86a98053e48e403570120d592dd09c60f5b4af625161c17c0
Oct 30 19:45:01.553143 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:01.553112    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889825-gkjfd"]
Oct 30 19:45:01.557615 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:45:01.557591    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbba9bd45_e2ea_4652_a1bf_28245075792a.slice/crio-e3b6f18809fa126e9a0dfaf16579312e5bb2e4d4a78c0620b7d2f5780cc284da WatchSource:0}: Error finding container e3b6f18809fa126e9a0dfaf16579312e5bb2e4d4a78c0620b7d2f5780cc284da: Status 404 returned error can't find the container with id e3b6f18809fa126e9a0dfaf16579312e5bb2e4d4a78c0620b7d2f5780cc284da
Oct 30 19:45:01.595316 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:01.595239    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5"]
Oct 30 19:45:01.662537 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:01.662507    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn"]
Oct 30 19:45:01.714905 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:45:01.714780    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dc2448a_45e5_4f3c_9458_ad45df73bf67.slice/crio-c70acc65bb47f13ed654d6dad367a8f6f25c6d249c209f8e53a7b7267ab9ce60 WatchSource:0}: Error finding container c70acc65bb47f13ed654d6dad367a8f6f25c6d249c209f8e53a7b7267ab9ce60: Status 404 returned error can't find the container with id c70acc65bb47f13ed654d6dad367a8f6f25c6d249c209f8e53a7b7267ab9ce60
Oct 30 19:45:01.748580 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:01.748551    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw"]
Oct 30 19:45:01.817420 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:01.817393    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j" event={"ID":"c8186fe3-60be-4434-aff7-e8251491d6c9","Type":"ContainerStarted","Data":"d444eb21ae21da010d0c8ae56a48c1abd49ca0de5840310360b6d2646778f76b"}
Oct 30 19:45:01.819081 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:01.819053    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-gkjfd" event={"ID":"bba9bd45-e2ea-4652-a1bf-28245075792a","Type":"ContainerStarted","Data":"e3b6f18809fa126e9a0dfaf16579312e5bb2e4d4a78c0620b7d2f5780cc284da"}
Oct 30 19:45:01.820458 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:01.820432    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5" event={"ID":"a5607fe2-1ded-4e25-8361-baa33c565d00","Type":"ContainerStarted","Data":"bb1b11ccf2f8bbbe02c2c602ecbc4c6de8b37182dbf24d23f1231c0242fed979"}
Oct 30 19:45:01.821828 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:01.821803    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn" event={"ID":"2dc2448a-45e5-4f3c-9458-ad45df73bf67","Type":"ContainerStarted","Data":"c70acc65bb47f13ed654d6dad367a8f6f25c6d249c209f8e53a7b7267ab9ce60"}
Oct 30 19:45:01.823346 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:01.823286    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp" event={"ID":"4353f4bc-7a39-4df3-82dd-b6b58ef16b09","Type":"ContainerStarted","Data":"8f654f0715d16e2280e89019de1ab7ec208f19a2e8f9f6b66f7ff80efa835e6a"}
Oct 30 19:45:01.825131 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:01.825101    5907 generic.go:356] "Generic (PLEG): container finished" podID="d925ef89-ce40-4ed1-ad1a-eadd4778245e" containerID="6fd5df0e49899c37be4b643fdbc1e795b2606432a7252a19ba46e5eac073385f" exitCode=0
Oct 30 19:45:01.825306 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:01.825273    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-086f2a-9b5f7cf-z2579" event={"ID":"d925ef89-ce40-4ed1-ad1a-eadd4778245e","Type":"ContainerDied","Data":"6fd5df0e49899c37be4b643fdbc1e795b2606432a7252a19ba46e5eac073385f"}
Oct 30 19:45:01.848233 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:01.848192    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9" event={"ID":"982777a9-020a-4038-a0a8-c3b0fbef7bc2","Type":"ContainerStarted","Data":"1b4d5332c1bfd9b86a98053e48e403570120d592dd09c60f5b4af625161c17c0"}
Oct 30 19:45:01.856524 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:01.856501    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-086f2a-9b5f7cf-z2579"
Oct 30 19:45:01.911500 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:01.910273    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d925ef89-ce40-4ed1-ad1a-eadd4778245e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d925ef89-ce40-4ed1-ad1a-eadd4778245e-plugins-config-volume\") pod \"d925ef89-ce40-4ed1-ad1a-eadd4778245e\" (UID: \"d925ef89-ce40-4ed1-ad1a-eadd4778245e\") "
Oct 30 19:45:01.911500 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:01.910662    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d925ef89-ce40-4ed1-ad1a-eadd4778245e-kube-api-access-56b5t\" (UniqueName: \"kubernetes.io/projected/d925ef89-ce40-4ed1-ad1a-eadd4778245e-kube-api-access-56b5t\") pod \"d925ef89-ce40-4ed1-ad1a-eadd4778245e\" (UID: \"d925ef89-ce40-4ed1-ad1a-eadd4778245e\") "
Oct 30 19:45:01.911500 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:01.911248    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d925ef89-ce40-4ed1-ad1a-eadd4778245e-plugins-config-volume" pod "d925ef89-ce40-4ed1-ad1a-eadd4778245e" (UID: "d925ef89-ce40-4ed1-ad1a-eadd4778245e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:01.911803 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:01.911769    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d925ef89-ce40-4ed1-ad1a-eadd4778245e-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:02.013472 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:02.013382    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d925ef89-ce40-4ed1-ad1a-eadd4778245e-kube-api-access-56b5t" pod "d925ef89-ce40-4ed1-ad1a-eadd4778245e" (UID: "d925ef89-ce40-4ed1-ad1a-eadd4778245e"). InnerVolumeSpecName "kube-api-access-56b5t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:02.115292 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:02.114917    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56b5t\" (UniqueName: \"kubernetes.io/projected/d925ef89-ce40-4ed1-ad1a-eadd4778245e-kube-api-access-56b5t\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:02.292520 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:02.292440    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9e9a9805-721e-4939-a8c4-d078d6a3dc32" path="/var/lib/kubelet/pods/9e9a9805-721e-4939-a8c4-d078d6a3dc32/volumes"
Oct 30 19:45:02.859360 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:02.859320    5907 generic.go:356] "Generic (PLEG): container finished" podID="982777a9-020a-4038-a0a8-c3b0fbef7bc2" containerID="ea60e40ff62c88286e677c02c44e9669d5214f3273de847260cbfb89f5a99bcf" exitCode=60
Oct 30 19:45:02.859553 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:02.859375    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9" event={"ID":"982777a9-020a-4038-a0a8-c3b0fbef7bc2","Type":"ContainerDied","Data":"ea60e40ff62c88286e677c02c44e9669d5214f3273de847260cbfb89f5a99bcf"}
Oct 30 19:45:02.859765 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:02.859747    5907 scope.go:122] "RemoveContainer" containerID="ea60e40ff62c88286e677c02c44e9669d5214f3273de847260cbfb89f5a99bcf"
Oct 30 19:45:02.861279 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:02.861256    5907 generic.go:356] "Generic (PLEG): container finished" podID="c8186fe3-60be-4434-aff7-e8251491d6c9" containerID="f643f22f75079326c8b610599aa16fb8a9e3eee732dcc84bf7604a96966958ee" exitCode=60
Oct 30 19:45:02.861396 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:02.861334    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j" event={"ID":"c8186fe3-60be-4434-aff7-e8251491d6c9","Type":"ContainerDied","Data":"f643f22f75079326c8b610599aa16fb8a9e3eee732dcc84bf7604a96966958ee"}
Oct 30 19:45:02.861730 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:02.861715    5907 scope.go:122] "RemoveContainer" containerID="f643f22f75079326c8b610599aa16fb8a9e3eee732dcc84bf7604a96966958ee"
Oct 30 19:45:02.863354 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:02.863330    5907 generic.go:356] "Generic (PLEG): container finished" podID="bba9bd45-e2ea-4652-a1bf-28245075792a" containerID="b9c325c608ca014ac014a8581307a0a26c6f05fe2ee8a87c627a4aa88c5eaee8" exitCode=60
Oct 30 19:45:02.863432 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:02.863422    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-gkjfd" event={"ID":"bba9bd45-e2ea-4652-a1bf-28245075792a","Type":"ContainerDied","Data":"b9c325c608ca014ac014a8581307a0a26c6f05fe2ee8a87c627a4aa88c5eaee8"}
Oct 30 19:45:02.863620 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:02.863601    5907 scope.go:122] "RemoveContainer" containerID="b9c325c608ca014ac014a8581307a0a26c6f05fe2ee8a87c627a4aa88c5eaee8"
Oct 30 19:45:02.865596 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:02.865578    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5" event={"ID":"a5607fe2-1ded-4e25-8361-baa33c565d00","Type":"ContainerStarted","Data":"fc7a6da970ab39aecef7096f58c7cfcda4411f29192339d1eece596c8caebb2a"}
Oct 30 19:45:02.867006 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:02.866986    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn" event={"ID":"2dc2448a-45e5-4f3c-9458-ad45df73bf67","Type":"ContainerStarted","Data":"d0323ddeaf9f83222305437fa644a9738dbc16590bf090635a6b952ddef6d679"}
Oct 30 19:45:02.868482 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:02.868458    5907 generic.go:356] "Generic (PLEG): container finished" podID="4353f4bc-7a39-4df3-82dd-b6b58ef16b09" containerID="28e754ff105c0aa7118ef99dde7bd884bca225aa54d8f00dccad997eb51fbadc" exitCode=60
Oct 30 19:45:02.868591 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:02.868543    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp" event={"ID":"4353f4bc-7a39-4df3-82dd-b6b58ef16b09","Type":"ContainerDied","Data":"28e754ff105c0aa7118ef99dde7bd884bca225aa54d8f00dccad997eb51fbadc"}
Oct 30 19:45:02.868894 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:02.868719    5907 scope.go:122] "RemoveContainer" containerID="28e754ff105c0aa7118ef99dde7bd884bca225aa54d8f00dccad997eb51fbadc"
Oct 30 19:45:02.871293 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:02.871264    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-086f2a-9b5f7cf-z2579"
Oct 30 19:45:02.871293 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:02.871270    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-086f2a-9b5f7cf-z2579" event={"ID":"d925ef89-ce40-4ed1-ad1a-eadd4778245e","Type":"ContainerDied","Data":"e5413a4b665606c39fa9a4abaf5ba4f73788c178d5df26014f7946e29d1ca4dc"}
Oct 30 19:45:02.871461 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:02.871311    5907 scope.go:122] "RemoveContainer" containerID="6fd5df0e49899c37be4b643fdbc1e795b2606432a7252a19ba46e5eac073385f"
Oct 30 19:45:02.873089 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:02.873067    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw" event={"ID":"5dbb34bb-6526-4dc5-bf66-9850efbdca88","Type":"ContainerStarted","Data":"bb4fa91de0ed49cb196aeeff051867e965712be69b29e2778615787fe24765d3"}
Oct 30 19:45:02.873184 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:02.873111    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw" event={"ID":"5dbb34bb-6526-4dc5-bf66-9850efbdca88","Type":"ContainerStarted","Data":"c67b661ba78475d2564397c024e29cb9771aa4f7e1bb7f2200e9a125341fd626"}
Oct 30 19:45:03.019837 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:03.019750    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5" podStartSLOduration=3.019726435 podStartE2EDuration="3.019726435s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:02.991401064 +0000 UTC m=+1414.834333002" watchObservedRunningTime="2026-10-30 19:45:03.019726435 +0000 UTC m=+1414.862658376"
Oct 30 19:45:03.021668 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:03.020724    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw" podStartSLOduration=3.020710723 podStartE2EDuration="3.020710723s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:02.942272305 +0000 UTC m=+1414.785204243" watchObservedRunningTime="2026-10-30 19:45:03.020710723 +0000 UTC m=+1414.863642686"
Oct 30 19:45:03.107425 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:03.107368    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn" podStartSLOduration=3.107350203 podStartE2EDuration="3.107350203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:03.104528592 +0000 UTC m=+1414.947460548" watchObservedRunningTime="2026-10-30 19:45:03.107350203 +0000 UTC m=+1414.950282143"
Oct 30 19:45:03.260388 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:03.260352    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-086f2a-9b5f7cf-z2579"]
Oct 30 19:45:03.264964 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:03.264938    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-086f2a-9b5f7cf-z2579"]
Oct 30 19:45:03.883775 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:03.883730    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j" event={"ID":"c8186fe3-60be-4434-aff7-e8251491d6c9","Type":"ContainerStarted","Data":"a6de9e6eccf0cb77d20ce4afda2228f9362e0e580b190a6c52b44da8d2e7d998"}
Oct 30 19:45:03.885935 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:03.885903    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-gkjfd" event={"ID":"bba9bd45-e2ea-4652-a1bf-28245075792a","Type":"ContainerStarted","Data":"0190e60c16190a9306490619027e7e934dd3c07feffff020cdb338bf48570745"}
Oct 30 19:45:03.887508 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:03.887484    5907 generic.go:356] "Generic (PLEG): container finished" podID="a5607fe2-1ded-4e25-8361-baa33c565d00" containerID="fc7a6da970ab39aecef7096f58c7cfcda4411f29192339d1eece596c8caebb2a" exitCode=60
Oct 30 19:45:03.887640 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:03.887555    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5" event={"ID":"a5607fe2-1ded-4e25-8361-baa33c565d00","Type":"ContainerDied","Data":"fc7a6da970ab39aecef7096f58c7cfcda4411f29192339d1eece596c8caebb2a"}
Oct 30 19:45:03.887988 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:03.887799    5907 scope.go:122] "RemoveContainer" containerID="fc7a6da970ab39aecef7096f58c7cfcda4411f29192339d1eece596c8caebb2a"
Oct 30 19:45:03.889184 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:03.889164    5907 generic.go:356] "Generic (PLEG): container finished" podID="2dc2448a-45e5-4f3c-9458-ad45df73bf67" containerID="d0323ddeaf9f83222305437fa644a9738dbc16590bf090635a6b952ddef6d679" exitCode=60
Oct 30 19:45:03.889309 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:03.889235    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn" event={"ID":"2dc2448a-45e5-4f3c-9458-ad45df73bf67","Type":"ContainerDied","Data":"d0323ddeaf9f83222305437fa644a9738dbc16590bf090635a6b952ddef6d679"}
Oct 30 19:45:03.889459 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:03.889429    5907 scope.go:122] "RemoveContainer" containerID="d0323ddeaf9f83222305437fa644a9738dbc16590bf090635a6b952ddef6d679"
Oct 30 19:45:03.891578 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:03.891540    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp" event={"ID":"4353f4bc-7a39-4df3-82dd-b6b58ef16b09","Type":"ContainerStarted","Data":"ac41ee4219efb1094c4ad9fcd632e618dc44cfb3dd705be8b8a69bc3e9d59356"}
Oct 30 19:45:03.894469 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:03.894445    5907 generic.go:356] "Generic (PLEG): container finished" podID="5dbb34bb-6526-4dc5-bf66-9850efbdca88" containerID="bb4fa91de0ed49cb196aeeff051867e965712be69b29e2778615787fe24765d3" exitCode=60
Oct 30 19:45:03.894575 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:03.894539    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw" event={"ID":"5dbb34bb-6526-4dc5-bf66-9850efbdca88","Type":"ContainerDied","Data":"bb4fa91de0ed49cb196aeeff051867e965712be69b29e2778615787fe24765d3"}
Oct 30 19:45:03.894730 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:03.894712    5907 scope.go:122] "RemoveContainer" containerID="bb4fa91de0ed49cb196aeeff051867e965712be69b29e2778615787fe24765d3"
Oct 30 19:45:03.896448 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:03.896428    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9" event={"ID":"982777a9-020a-4038-a0a8-c3b0fbef7bc2","Type":"ContainerStarted","Data":"2ae6ef3ab01b7899e1592b8b75d4c1e521faae45705642dc115c981b75c940cd"}
Oct 30 19:45:03.943037 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:03.942991    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j" podStartSLOduration=3.942977969 podStartE2EDuration="3.942977969s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:03.937971559 +0000 UTC m=+1415.780903499" watchObservedRunningTime="2026-10-30 19:45:03.942977969 +0000 UTC m=+1415.785909907"
Oct 30 19:45:04.125200 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:04.125137    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp" podStartSLOduration=4.125118838 podStartE2EDuration="4.125118838s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:04.117915377 +0000 UTC m=+1415.960847318" watchObservedRunningTime="2026-10-30 19:45:04.125118838 +0000 UTC m=+1415.968050778"
Oct 30 19:45:04.237083 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:04.237033    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9" podStartSLOduration=4.237016386 podStartE2EDuration="4.237016386s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:04.234332307 +0000 UTC m=+1416.077264245" watchObservedRunningTime="2026-10-30 19:45:04.237016386 +0000 UTC m=+1416.079948324"
Oct 30 19:45:04.290232 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:04.290197    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d925ef89-ce40-4ed1-ad1a-eadd4778245e" path="/var/lib/kubelet/pods/d925ef89-ce40-4ed1-ad1a-eadd4778245e/volumes"
Oct 30 19:45:04.299807 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:04.299752    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-gkjfd" podStartSLOduration=4.299736776 podStartE2EDuration="4.299736776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:04.297628839 +0000 UTC m=+1416.140560779" watchObservedRunningTime="2026-10-30 19:45:04.299736776 +0000 UTC m=+1416.142668720"
Oct 30 19:45:04.971429 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:04.971371    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn" event={"ID":"2dc2448a-45e5-4f3c-9458-ad45df73bf67","Type":"ContainerStarted","Data":"cad5bf566530385cd5073a63a62faf72347760ca81352113d10606241443ee4f"}
Oct 30 19:45:04.975798 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:04.975757    5907 generic.go:356] "Generic (PLEG): container finished" podID="4353f4bc-7a39-4df3-82dd-b6b58ef16b09" containerID="ac41ee4219efb1094c4ad9fcd632e618dc44cfb3dd705be8b8a69bc3e9d59356" exitCode=60
Oct 30 19:45:04.976155 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:04.976116    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp" event={"ID":"4353f4bc-7a39-4df3-82dd-b6b58ef16b09","Type":"ContainerDied","Data":"ac41ee4219efb1094c4ad9fcd632e618dc44cfb3dd705be8b8a69bc3e9d59356"}
Oct 30 19:45:04.976387 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:04.976357    5907 scope.go:122] "RemoveContainer" containerID="28e754ff105c0aa7118ef99dde7bd884bca225aa54d8f00dccad997eb51fbadc"
Oct 30 19:45:04.977097 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:04.977064    5907 scope.go:122] "RemoveContainer" containerID="ac41ee4219efb1094c4ad9fcd632e618dc44cfb3dd705be8b8a69bc3e9d59356"
Oct 30 19:45:04.977714 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:45:04.977677    5907 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp_odh-ai-gateway-infra(4353f4bc-7a39-4df3-82dd-b6b58ef16b09)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp" podUID="4353f4bc-7a39-4df3-82dd-b6b58ef16b09"
Oct 30 19:45:05.007388 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:05.002169    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw" event={"ID":"5dbb34bb-6526-4dc5-bf66-9850efbdca88","Type":"ContainerStarted","Data":"77a8ec3f9f68e41ebbd76133afe4721f9633c569306c703776b4206592e37648"}
Oct 30 19:45:05.036442 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:05.027897    5907 generic.go:356] "Generic (PLEG): container finished" podID="982777a9-020a-4038-a0a8-c3b0fbef7bc2" containerID="2ae6ef3ab01b7899e1592b8b75d4c1e521faae45705642dc115c981b75c940cd" exitCode=60
Oct 30 19:45:05.036442 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:05.028770    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9" event={"ID":"982777a9-020a-4038-a0a8-c3b0fbef7bc2","Type":"ContainerDied","Data":"2ae6ef3ab01b7899e1592b8b75d4c1e521faae45705642dc115c981b75c940cd"}
Oct 30 19:45:05.036442 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:05.029640    5907 scope.go:122] "RemoveContainer" containerID="2ae6ef3ab01b7899e1592b8b75d4c1e521faae45705642dc115c981b75c940cd"
Oct 30 19:45:05.036442 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:45:05.030296    5907 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9_odh-ai-gateway-infra(982777a9-020a-4038-a0a8-c3b0fbef7bc2)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9" podUID="982777a9-020a-4038-a0a8-c3b0fbef7bc2"
Oct 30 19:45:05.036442 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:05.034367    5907 generic.go:356] "Generic (PLEG): container finished" podID="c8186fe3-60be-4434-aff7-e8251491d6c9" containerID="a6de9e6eccf0cb77d20ce4afda2228f9362e0e580b190a6c52b44da8d2e7d998" exitCode=60
Oct 30 19:45:05.036442 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:05.034534    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j" event={"ID":"c8186fe3-60be-4434-aff7-e8251491d6c9","Type":"ContainerDied","Data":"a6de9e6eccf0cb77d20ce4afda2228f9362e0e580b190a6c52b44da8d2e7d998"}
Oct 30 19:45:05.036442 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:05.036020    5907 scope.go:122] "RemoveContainer" containerID="a6de9e6eccf0cb77d20ce4afda2228f9362e0e580b190a6c52b44da8d2e7d998"
Oct 30 19:45:05.041100 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:05.039926    5907 generic.go:356] "Generic (PLEG): container finished" podID="bba9bd45-e2ea-4652-a1bf-28245075792a" containerID="0190e60c16190a9306490619027e7e934dd3c07feffff020cdb338bf48570745" exitCode=60
Oct 30 19:45:05.041100 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:05.040034    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-gkjfd" event={"ID":"bba9bd45-e2ea-4652-a1bf-28245075792a","Type":"ContainerDied","Data":"0190e60c16190a9306490619027e7e934dd3c07feffff020cdb338bf48570745"}
Oct 30 19:45:05.059618 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:45:05.054570    5907 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j_odh-ai-gateway-infra(c8186fe3-60be-4434-aff7-e8251491d6c9)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j" podUID="c8186fe3-60be-4434-aff7-e8251491d6c9"
Oct 30 19:45:05.061036 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:05.061010    5907 scope.go:122] "RemoveContainer" containerID="0190e60c16190a9306490619027e7e934dd3c07feffff020cdb338bf48570745"
Oct 30 19:45:05.065811 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:45:05.064481    5907 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889825-gkjfd_odh-ai-gateway-infra(bba9bd45-e2ea-4652-a1bf-28245075792a)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-gkjfd" podUID="bba9bd45-e2ea-4652-a1bf-28245075792a"
Oct 30 19:45:05.065978 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:05.065948    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5" event={"ID":"a5607fe2-1ded-4e25-8361-baa33c565d00","Type":"ContainerStarted","Data":"7b5ff0aee7629e7449a3fd87814739a001283b0254e8ec9fcb543efb79e87252"}
Oct 30 19:45:05.157139 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:05.157118    5907 scope.go:122] "RemoveContainer" containerID="ea60e40ff62c88286e677c02c44e9669d5214f3273de847260cbfb89f5a99bcf"
Oct 30 19:45:05.172404 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:05.172381    5907 scope.go:122] "RemoveContainer" containerID="f643f22f75079326c8b610599aa16fb8a9e3eee732dcc84bf7604a96966958ee"
Oct 30 19:45:05.187590 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:05.187564    5907 scope.go:122] "RemoveContainer" containerID="b9c325c608ca014ac014a8581307a0a26c6f05fe2ee8a87c627a4aa88c5eaee8"
Oct 30 19:45:06.083651 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:06.083612    5907 scope.go:122] "RemoveContainer" containerID="0190e60c16190a9306490619027e7e934dd3c07feffff020cdb338bf48570745"
Oct 30 19:45:06.084336 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:45:06.084298    5907 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889825-gkjfd_odh-ai-gateway-infra(bba9bd45-e2ea-4652-a1bf-28245075792a)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-gkjfd" podUID="bba9bd45-e2ea-4652-a1bf-28245075792a"
Oct 30 19:45:06.095923 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:06.095845    5907 generic.go:356] "Generic (PLEG): container finished" podID="a5607fe2-1ded-4e25-8361-baa33c565d00" containerID="7b5ff0aee7629e7449a3fd87814739a001283b0254e8ec9fcb543efb79e87252" exitCode=60
Oct 30 19:45:06.096109 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:06.096000    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5" event={"ID":"a5607fe2-1ded-4e25-8361-baa33c565d00","Type":"ContainerDied","Data":"7b5ff0aee7629e7449a3fd87814739a001283b0254e8ec9fcb543efb79e87252"}
Oct 30 19:45:06.096109 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:06.096067    5907 scope.go:122] "RemoveContainer" containerID="fc7a6da970ab39aecef7096f58c7cfcda4411f29192339d1eece596c8caebb2a"
Oct 30 19:45:06.096463 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:06.096447    5907 scope.go:122] "RemoveContainer" containerID="7b5ff0aee7629e7449a3fd87814739a001283b0254e8ec9fcb543efb79e87252"
Oct 30 19:45:06.096767 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:45:06.096748    5907 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5_odh-ai-gateway-infra(a5607fe2-1ded-4e25-8361-baa33c565d00)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5" podUID="a5607fe2-1ded-4e25-8361-baa33c565d00"
Oct 30 19:45:06.110640 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:06.110612    5907 generic.go:356] "Generic (PLEG): container finished" podID="2dc2448a-45e5-4f3c-9458-ad45df73bf67" containerID="cad5bf566530385cd5073a63a62faf72347760ca81352113d10606241443ee4f" exitCode=60
Oct 30 19:45:06.110967 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:06.110929    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn" event={"ID":"2dc2448a-45e5-4f3c-9458-ad45df73bf67","Type":"ContainerDied","Data":"cad5bf566530385cd5073a63a62faf72347760ca81352113d10606241443ee4f"}
Oct 30 19:45:06.111610 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:06.111580    5907 scope.go:122] "RemoveContainer" containerID="cad5bf566530385cd5073a63a62faf72347760ca81352113d10606241443ee4f"
Oct 30 19:45:06.112002 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:45:06.111934    5907 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn_odh-ai-gateway-infra(2dc2448a-45e5-4f3c-9458-ad45df73bf67)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn" podUID="2dc2448a-45e5-4f3c-9458-ad45df73bf67"
Oct 30 19:45:06.114991 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:06.114960    5907 scope.go:122] "RemoveContainer" containerID="ac41ee4219efb1094c4ad9fcd632e618dc44cfb3dd705be8b8a69bc3e9d59356"
Oct 30 19:45:06.115276 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:45:06.115254    5907 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp_odh-ai-gateway-infra(4353f4bc-7a39-4df3-82dd-b6b58ef16b09)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp" podUID="4353f4bc-7a39-4df3-82dd-b6b58ef16b09"
Oct 30 19:45:06.116091 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:06.116036    5907 scope.go:122] "RemoveContainer" containerID="d0323ddeaf9f83222305437fa644a9738dbc16590bf090635a6b952ddef6d679"
Oct 30 19:45:06.118592 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:06.118519    5907 generic.go:356] "Generic (PLEG): container finished" podID="5dbb34bb-6526-4dc5-bf66-9850efbdca88" containerID="77a8ec3f9f68e41ebbd76133afe4721f9633c569306c703776b4206592e37648" exitCode=60
Oct 30 19:45:06.118749 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:06.118729    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw" event={"ID":"5dbb34bb-6526-4dc5-bf66-9850efbdca88","Type":"ContainerDied","Data":"77a8ec3f9f68e41ebbd76133afe4721f9633c569306c703776b4206592e37648"}
Oct 30 19:45:06.119119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:06.119100    5907 scope.go:122] "RemoveContainer" containerID="77a8ec3f9f68e41ebbd76133afe4721f9633c569306c703776b4206592e37648"
Oct 30 19:45:06.119381 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:45:06.119359    5907 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw_odh-ai-gateway-infra(5dbb34bb-6526-4dc5-bf66-9850efbdca88)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw" podUID="5dbb34bb-6526-4dc5-bf66-9850efbdca88"
Oct 30 19:45:06.122053 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:06.122033    5907 scope.go:122] "RemoveContainer" containerID="2ae6ef3ab01b7899e1592b8b75d4c1e521faae45705642dc115c981b75c940cd"
Oct 30 19:45:06.124947 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:45:06.122293    5907 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9_odh-ai-gateway-infra(982777a9-020a-4038-a0a8-c3b0fbef7bc2)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9" podUID="982777a9-020a-4038-a0a8-c3b0fbef7bc2"
Oct 30 19:45:06.125219 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:06.125197    5907 scope.go:122] "RemoveContainer" containerID="a6de9e6eccf0cb77d20ce4afda2228f9362e0e580b190a6c52b44da8d2e7d998"
Oct 30 19:45:06.125522 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:45:06.125486    5907 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j_odh-ai-gateway-infra(c8186fe3-60be-4434-aff7-e8251491d6c9)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j" podUID="c8186fe3-60be-4434-aff7-e8251491d6c9"
Oct 30 19:45:06.212080 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:06.206046    5907 scope.go:122] "RemoveContainer" containerID="bb4fa91de0ed49cb196aeeff051867e965712be69b29e2778615787fe24765d3"
Oct 30 19:45:07.154309 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:07.154277    5907 scope.go:122] "RemoveContainer" containerID="7b5ff0aee7629e7449a3fd87814739a001283b0254e8ec9fcb543efb79e87252"
Oct 30 19:45:07.154728 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:45:07.154692    5907 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5_odh-ai-gateway-infra(a5607fe2-1ded-4e25-8361-baa33c565d00)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5" podUID="a5607fe2-1ded-4e25-8361-baa33c565d00"
Oct 30 19:45:07.156099 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:07.156078    5907 scope.go:122] "RemoveContainer" containerID="cad5bf566530385cd5073a63a62faf72347760ca81352113d10606241443ee4f"
Oct 30 19:45:07.156441 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:45:07.156416    5907 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn_odh-ai-gateway-infra(2dc2448a-45e5-4f3c-9458-ad45df73bf67)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn" podUID="2dc2448a-45e5-4f3c-9458-ad45df73bf67"
Oct 30 19:45:07.157739 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:07.157722    5907 scope.go:122] "RemoveContainer" containerID="77a8ec3f9f68e41ebbd76133afe4721f9633c569306c703776b4206592e37648"
Oct 30 19:45:07.158066 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:45:07.158047    5907 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw_odh-ai-gateway-infra(5dbb34bb-6526-4dc5-bf66-9850efbdca88)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw" podUID="5dbb34bb-6526-4dc5-bf66-9850efbdca88"
Oct 30 19:45:07.262109 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:07.262076    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-mhv64"]
Oct 30 19:45:07.262977 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:07.262958    5907 cpu_manager.go:414] "containerMap: removing container" podUID="d925ef89-ce40-4ed1-ad1a-eadd4778245e" containerName="payload-pre-processing"
Oct 30 19:45:07.263044 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:07.262979    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d925ef89-ce40-4ed1-ad1a-eadd4778245e" containerName="payload-pre-processing"
Oct 30 19:45:07.263117 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:07.263107    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="d925ef89-ce40-4ed1-ad1a-eadd4778245e" containerName="payload-pre-processing"
Oct 30 19:45:07.268490 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:07.268470    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-mhv64"
Oct 30 19:45:07.293953 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:07.293925    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-67953efd\""
Oct 30 19:45:07.294108 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:07.294053    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-67953efd-dockercfg-bc54t\""
Oct 30 19:45:07.294955 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:07.294927    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-mhv64"]
Oct 30 19:45:07.538942 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:07.530127    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c3bc483-1c8c-43d7-a336-ee6e04eda24b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-67953efd-6797bd4db7-mhv64\" (UID: \"6c3bc483-1c8c-43d7-a336-ee6e04eda24b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-mhv64"
Oct 30 19:45:07.538942 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:07.530415    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnx6d\" (UniqueName: \"kubernetes.io/projected/6c3bc483-1c8c-43d7-a336-ee6e04eda24b-kube-api-access-pnx6d\") pod \"payload-pre-processing-e2e-mt-67953efd-6797bd4db7-mhv64\" (UID: \"6c3bc483-1c8c-43d7-a336-ee6e04eda24b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-mhv64"
Oct 30 19:45:07.584886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:07.576428    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-67953efd-b8797cccd-pr7bp"]
Oct 30 19:45:07.643950 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:07.643731    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnx6d\" (UniqueName: \"kubernetes.io/projected/6c3bc483-1c8c-43d7-a336-ee6e04eda24b-kube-api-access-pnx6d\") pod \"payload-pre-processing-e2e-mt-67953efd-6797bd4db7-mhv64\" (UID: \"6c3bc483-1c8c-43d7-a336-ee6e04eda24b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-mhv64"
Oct 30 19:45:07.644160 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:07.644062    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c3bc483-1c8c-43d7-a336-ee6e04eda24b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-67953efd-6797bd4db7-mhv64\" (UID: \"6c3bc483-1c8c-43d7-a336-ee6e04eda24b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-mhv64"
Oct 30 19:45:07.666581 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:07.645218    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c3bc483-1c8c-43d7-a336-ee6e04eda24b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-67953efd-6797bd4db7-mhv64\" (UID: \"6c3bc483-1c8c-43d7-a336-ee6e04eda24b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-mhv64"
Oct 30 19:45:07.667295 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:07.667258    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-67953efd-b8797cccd-pr7bp"
Oct 30 19:45:07.744003 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:07.743910    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-67953efd-b8797cccd-pr7bp"]
Oct 30 19:45:07.765552 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:07.745313    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdhfv\" (UniqueName: \"kubernetes.io/projected/5dfc7248-d1a7-4093-a2b8-eaf603d9a660-kube-api-access-vdhfv\") pod \"payload-processing-e2e-mt-67953efd-b8797cccd-pr7bp\" (UID: \"5dfc7248-d1a7-4093-a2b8-eaf603d9a660\") " pod="openshift-ingress/payload-processing-e2e-mt-67953efd-b8797cccd-pr7bp"
Oct 30 19:45:07.765552 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:07.745684    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5dfc7248-d1a7-4093-a2b8-eaf603d9a660-plugins-config-volume\") pod \"payload-processing-e2e-mt-67953efd-b8797cccd-pr7bp\" (UID: \"5dfc7248-d1a7-4093-a2b8-eaf603d9a660\") " pod="openshift-ingress/payload-processing-e2e-mt-67953efd-b8797cccd-pr7bp"
Oct 30 19:45:07.780886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:07.768363    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnx6d\" (UniqueName: \"kubernetes.io/projected/6c3bc483-1c8c-43d7-a336-ee6e04eda24b-kube-api-access-pnx6d\") pod \"payload-pre-processing-e2e-mt-67953efd-6797bd4db7-mhv64\" (UID: \"6c3bc483-1c8c-43d7-a336-ee6e04eda24b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-mhv64"
Oct 30 19:45:07.847689 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:07.847588    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5dfc7248-d1a7-4093-a2b8-eaf603d9a660-plugins-config-volume\") pod \"payload-processing-e2e-mt-67953efd-b8797cccd-pr7bp\" (UID: \"5dfc7248-d1a7-4093-a2b8-eaf603d9a660\") " pod="openshift-ingress/payload-processing-e2e-mt-67953efd-b8797cccd-pr7bp"
Oct 30 19:45:07.847895 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:07.847693    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdhfv\" (UniqueName: \"kubernetes.io/projected/5dfc7248-d1a7-4093-a2b8-eaf603d9a660-kube-api-access-vdhfv\") pod \"payload-processing-e2e-mt-67953efd-b8797cccd-pr7bp\" (UID: \"5dfc7248-d1a7-4093-a2b8-eaf603d9a660\") " pod="openshift-ingress/payload-processing-e2e-mt-67953efd-b8797cccd-pr7bp"
Oct 30 19:45:07.848390 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:07.848369    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5dfc7248-d1a7-4093-a2b8-eaf603d9a660-plugins-config-volume\") pod \"payload-processing-e2e-mt-67953efd-b8797cccd-pr7bp\" (UID: \"5dfc7248-d1a7-4093-a2b8-eaf603d9a660\") " pod="openshift-ingress/payload-processing-e2e-mt-67953efd-b8797cccd-pr7bp"
Oct 30 19:45:07.872665 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:07.872631    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdhfv\" (UniqueName: \"kubernetes.io/projected/5dfc7248-d1a7-4093-a2b8-eaf603d9a660-kube-api-access-vdhfv\") pod \"payload-processing-e2e-mt-67953efd-b8797cccd-pr7bp\" (UID: \"5dfc7248-d1a7-4093-a2b8-eaf603d9a660\") " pod="openshift-ingress/payload-processing-e2e-mt-67953efd-b8797cccd-pr7bp"
Oct 30 19:45:07.884382 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:07.884355    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-mhv64"
Oct 30 19:45:08.098704 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:08.098606    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-67953efd-b8797cccd-pr7bp"
Oct 30 19:45:08.257930 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:45:08.256846    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c3bc483_1c8c_43d7_a336_ee6e04eda24b.slice/crio-a418ef2c8351cf28071de2f39b9590c5a10e65258617c073b85b5a1ac7785b80 WatchSource:0}: Error finding container a418ef2c8351cf28071de2f39b9590c5a10e65258617c073b85b5a1ac7785b80: Status 404 returned error can't find the container with id a418ef2c8351cf28071de2f39b9590c5a10e65258617c073b85b5a1ac7785b80
Oct 30 19:45:08.260032 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:08.260002    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-mhv64"]
Oct 30 19:45:08.537914 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:08.531430    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-67953efd-b8797cccd-pr7bp"]
Oct 30 19:45:08.540961 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:45:08.540923    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dfc7248_d1a7_4093_a2b8_eaf603d9a660.slice/crio-4d7963ccf3e6f4fe59a86ff2b79c4b201fe23c87516ffeb68336b94774bbf63c WatchSource:0}: Error finding container 4d7963ccf3e6f4fe59a86ff2b79c4b201fe23c87516ffeb68336b94774bbf63c: Status 404 returned error can't find the container with id 4d7963ccf3e6f4fe59a86ff2b79c4b201fe23c87516ffeb68336b94774bbf63c
Oct 30 19:45:09.182238 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:09.182195    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-mhv64" event={"ID":"6c3bc483-1c8c-43d7-a336-ee6e04eda24b","Type":"ContainerStarted","Data":"1e9e1780e07cffd6e6cde29fd932f0dd4116fcbd121eca93ec309b19abb8f4e3"}
Oct 30 19:45:09.182467 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:09.182257    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-mhv64" event={"ID":"6c3bc483-1c8c-43d7-a336-ee6e04eda24b","Type":"ContainerStarted","Data":"a418ef2c8351cf28071de2f39b9590c5a10e65258617c073b85b5a1ac7785b80"}
Oct 30 19:45:09.183214 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:09.183189    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-mhv64"
Oct 30 19:45:09.185285 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:09.185252    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-67953efd-b8797cccd-pr7bp" event={"ID":"5dfc7248-d1a7-4093-a2b8-eaf603d9a660","Type":"ContainerStarted","Data":"494cdc5485790e982281a5fde03c47f87a4c17e3cdaf382d37d347e167732b60"}
Oct 30 19:45:09.185382 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:09.185315    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-67953efd-b8797cccd-pr7bp"
Oct 30 19:45:09.185426 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:09.185385    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-67953efd-b8797cccd-pr7bp" event={"ID":"5dfc7248-d1a7-4093-a2b8-eaf603d9a660","Type":"ContainerStarted","Data":"4d7963ccf3e6f4fe59a86ff2b79c4b201fe23c87516ffeb68336b94774bbf63c"}
Oct 30 19:45:09.247488 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:09.247431    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-mhv64" podStartSLOduration=2.247417771 podStartE2EDuration="2.247417771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:09.246168869 +0000 UTC m=+1421.089100807" watchObservedRunningTime="2026-10-30 19:45:09.247417771 +0000 UTC m=+1421.090349706"
Oct 30 19:45:09.338224 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:09.338160    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-67953efd-b8797cccd-pr7bp" podStartSLOduration=2.338140123 podStartE2EDuration="2.338140123s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:09.337729794 +0000 UTC m=+1421.180661736" watchObservedRunningTime="2026-10-30 19:45:09.338140123 +0000 UTC m=+1421.181072063"
Oct 30 19:45:09.854000 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:09.853965    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf"]
Oct 30 19:45:09.859160 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:09.859139    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf"
Oct 30 19:45:09.891081 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:09.891040    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf"]
Oct 30 19:45:09.902415 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:09.902392    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-9de441-serving-cert\""
Oct 30 19:45:09.997528 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:09.997486    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b021bfa-85df-4d44-8717-3c99b0ea7ae1-maas-api-tls\") pod \"maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf\" (UID: \"7b021bfa-85df-4d44-8717-3c99b0ea7ae1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf"
Oct 30 19:45:09.997715 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:09.997545    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b021bfa-85df-4d44-8717-3c99b0ea7ae1-metrics-certs\") pod \"maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf\" (UID: \"7b021bfa-85df-4d44-8717-3c99b0ea7ae1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf"
Oct 30 19:45:09.997715 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:09.997680    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b021bfa-85df-4d44-8717-3c99b0ea7ae1-openshift-service-ca\") pod \"maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf\" (UID: \"7b021bfa-85df-4d44-8717-3c99b0ea7ae1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf"
Oct 30 19:45:09.997945 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:09.997921    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9thwk\" (UniqueName: \"kubernetes.io/projected/7b021bfa-85df-4d44-8717-3c99b0ea7ae1-kube-api-access-9thwk\") pod \"maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf\" (UID: \"7b021bfa-85df-4d44-8717-3c99b0ea7ae1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf"
Oct 30 19:45:10.112935 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:10.108111    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b021bfa-85df-4d44-8717-3c99b0ea7ae1-maas-api-tls\") pod \"maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf\" (UID: \"7b021bfa-85df-4d44-8717-3c99b0ea7ae1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf"
Oct 30 19:45:10.112935 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:10.108770    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b021bfa-85df-4d44-8717-3c99b0ea7ae1-metrics-certs\") pod \"maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf\" (UID: \"7b021bfa-85df-4d44-8717-3c99b0ea7ae1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf"
Oct 30 19:45:10.112935 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:10.109459    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b021bfa-85df-4d44-8717-3c99b0ea7ae1-openshift-service-ca\") pod \"maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf\" (UID: \"7b021bfa-85df-4d44-8717-3c99b0ea7ae1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf"
Oct 30 19:45:10.119883 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:10.114576    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b021bfa-85df-4d44-8717-3c99b0ea7ae1-openshift-service-ca\") pod \"maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf\" (UID: \"7b021bfa-85df-4d44-8717-3c99b0ea7ae1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf"
Oct 30 19:45:10.119883 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:10.114937    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9thwk\" (UniqueName: \"kubernetes.io/projected/7b021bfa-85df-4d44-8717-3c99b0ea7ae1-kube-api-access-9thwk\") pod \"maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf\" (UID: \"7b021bfa-85df-4d44-8717-3c99b0ea7ae1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf"
Oct 30 19:45:10.137886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:10.133467    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b021bfa-85df-4d44-8717-3c99b0ea7ae1-maas-api-tls\") pod \"maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf\" (UID: \"7b021bfa-85df-4d44-8717-3c99b0ea7ae1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf"
Oct 30 19:45:10.149890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:10.146447    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b021bfa-85df-4d44-8717-3c99b0ea7ae1-metrics-certs\") pod \"maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf\" (UID: \"7b021bfa-85df-4d44-8717-3c99b0ea7ae1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf"
Oct 30 19:45:10.168882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:10.164016    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9thwk\" (UniqueName: \"kubernetes.io/projected/7b021bfa-85df-4d44-8717-3c99b0ea7ae1-kube-api-access-9thwk\") pod \"maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf\" (UID: \"7b021bfa-85df-4d44-8717-3c99b0ea7ae1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf"
Oct 30 19:45:10.180485 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:10.179206    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf"
Oct 30 19:45:10.547083 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:10.546610    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf"]
Oct 30 19:45:11.281226 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:11.281078    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf" event={"ID":"7b021bfa-85df-4d44-8717-3c99b0ea7ae1","Type":"ContainerStarted","Data":"877531ad62d78646b8439bfa6b8f8b371ad1857599185356d5238a5da6f5a97d"}
Oct 30 19:45:11.281226 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:11.281129    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf" event={"ID":"7b021bfa-85df-4d44-8717-3c99b0ea7ae1","Type":"ContainerStarted","Data":"3d18d533681716acfe4561861735cb3260527839d95f13dbc58397ce4b1fbb62"}
Oct 30 19:45:11.281226 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:11.281177    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf"
Oct 30 19:45:11.342036 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:11.341971    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf" podStartSLOduration=1.942985777 podStartE2EDuration="2.341955884s" totalImagesPullingTime="398.970107ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:11.330188493 +0000 UTC m=+1423.173120433" watchObservedRunningTime="2026-10-30 19:45:11.341955884 +0000 UTC m=+1423.184887885"
Oct 30 19:45:15.140005 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:15.139965    5907 patch_prober.go:29] interesting pod/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.199:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:45:15.140446 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:15.140030    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv" podUID="57838872-c482-41ec-bda0-4a517fb81dcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.199:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:16.289993 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:16.289966    5907 scope.go:122] "RemoveContainer" containerID="a6de9e6eccf0cb77d20ce4afda2228f9362e0e580b190a6c52b44da8d2e7d998"
Oct 30 19:45:16.290496 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:16.290060    5907 scope.go:122] "RemoveContainer" containerID="0190e60c16190a9306490619027e7e934dd3c07feffff020cdb338bf48570745"
Oct 30 19:45:16.348540 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:16.348509    5907 generic.go:356] "Generic (PLEG): container finished" podID="57838872-c482-41ec-bda0-4a517fb81dcd" containerID="1d1433dd49d5c61722711f2b46da6b97cb8aa4b0fc9e164520f3452b073521af" exitCode=0
Oct 30 19:45:16.348663 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:16.348580    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv" event={"ID":"57838872-c482-41ec-bda0-4a517fb81dcd","Type":"ContainerDied","Data":"1d1433dd49d5c61722711f2b46da6b97cb8aa4b0fc9e164520f3452b073521af"}
Oct 30 19:45:16.349082 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:16.349065    5907 scope.go:122] "RemoveContainer" containerID="1d1433dd49d5c61722711f2b46da6b97cb8aa4b0fc9e164520f3452b073521af"
Oct 30 19:45:16.350743 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:16.350715    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:45:16.350828 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:16.350785    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:45:16.350902 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:16.350829    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:45:17.283068 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:17.282978    5907 scope.go:122] "RemoveContainer" containerID="2ae6ef3ab01b7899e1592b8b75d4c1e521faae45705642dc115c981b75c940cd"
Oct 30 19:45:17.285342 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:17.285295    5907 scope.go:122] "RemoveContainer" containerID="ac41ee4219efb1094c4ad9fcd632e618dc44cfb3dd705be8b8a69bc3e9d59356"
Oct 30 19:45:17.398531 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:17.398489    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf"
Oct 30 19:45:17.408660 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:17.408589    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j" event={"ID":"c8186fe3-60be-4434-aff7-e8251491d6c9","Type":"ContainerStarted","Data":"f4554c762f52b80a1784aaa2c61e701db01035cae31cb293872a764ffda15af6"}
Oct 30 19:45:17.443890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:17.438002    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-gkjfd" event={"ID":"bba9bd45-e2ea-4652-a1bf-28245075792a","Type":"ContainerStarted","Data":"b6f6830faccdf8cf3ff609e71480e25261df25efbff6a8255c7fd6f47c2c9aa6"}
Oct 30 19:45:17.468421 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:17.468387    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv" event={"ID":"57838872-c482-41ec-bda0-4a517fb81dcd","Type":"ContainerStarted","Data":"7d5cbf1113d7cdb51c3e45581ca40732e044dbc3b8d173681cf18aedc4290c97"}
Oct 30 19:45:17.556475 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:45:17.556191    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:45:17.970065 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:17.970033    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:45:17.970273 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:17.970251    5907 patch_prober.go:29] interesting pod/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.199:15021/healthz/ready\": dial tcp 10.134.0.199:15021: connect: connection refused" start-of-body=
Oct 30 19:45:17.970327 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:17.970300    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv" podUID="57838872-c482-41ec-bda0-4a517fb81dcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.199:15021/healthz/ready\": dial tcp 10.134.0.199:15021: connect: connection refused"
Oct 30 19:45:18.285702 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:18.285666    5907 scope.go:122] "RemoveContainer" containerID="cad5bf566530385cd5073a63a62faf72347760ca81352113d10606241443ee4f"
Oct 30 19:45:18.285932 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:18.285835    5907 scope.go:122] "RemoveContainer" containerID="7b5ff0aee7629e7449a3fd87814739a001283b0254e8ec9fcb543efb79e87252"
Oct 30 19:45:18.479341 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:18.479302    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp" event={"ID":"4353f4bc-7a39-4df3-82dd-b6b58ef16b09","Type":"ContainerStarted","Data":"896a3e68cdd9948d82086172e9eeedb4f17d14039b04febaddd742b557b9f193"}
Oct 30 19:45:18.481499 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:18.481477    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9" event={"ID":"982777a9-020a-4038-a0a8-c3b0fbef7bc2","Type":"ContainerStarted","Data":"e2658c11d03525f72d47c4b212ea24b9b74c57b7562f4cbd01ec7f08c03deddf"}
Oct 30 19:45:18.483523 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:18.483501    5907 generic.go:356] "Generic (PLEG): container finished" podID="c8186fe3-60be-4434-aff7-e8251491d6c9" containerID="f4554c762f52b80a1784aaa2c61e701db01035cae31cb293872a764ffda15af6" exitCode=60
Oct 30 19:45:18.483636 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:18.483568    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j" event={"ID":"c8186fe3-60be-4434-aff7-e8251491d6c9","Type":"ContainerDied","Data":"f4554c762f52b80a1784aaa2c61e701db01035cae31cb293872a764ffda15af6"}
Oct 30 19:45:18.483636 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:18.483614    5907 scope.go:122] "RemoveContainer" containerID="a6de9e6eccf0cb77d20ce4afda2228f9362e0e580b190a6c52b44da8d2e7d998"
Oct 30 19:45:18.483976 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:18.483959    5907 scope.go:122] "RemoveContainer" containerID="f4554c762f52b80a1784aaa2c61e701db01035cae31cb293872a764ffda15af6"
Oct 30 19:45:18.484289 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:45:18.484266    5907 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j_odh-ai-gateway-infra(c8186fe3-60be-4434-aff7-e8251491d6c9)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j" podUID="c8186fe3-60be-4434-aff7-e8251491d6c9"
Oct 30 19:45:18.486063 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:18.486042    5907 generic.go:356] "Generic (PLEG): container finished" podID="bba9bd45-e2ea-4652-a1bf-28245075792a" containerID="b6f6830faccdf8cf3ff609e71480e25261df25efbff6a8255c7fd6f47c2c9aa6" exitCode=60
Oct 30 19:45:18.486160 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:18.486076    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-gkjfd" event={"ID":"bba9bd45-e2ea-4652-a1bf-28245075792a","Type":"ContainerDied","Data":"b6f6830faccdf8cf3ff609e71480e25261df25efbff6a8255c7fd6f47c2c9aa6"}
Oct 30 19:45:18.486562 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:18.486548    5907 scope.go:122] "RemoveContainer" containerID="b6f6830faccdf8cf3ff609e71480e25261df25efbff6a8255c7fd6f47c2c9aa6"
Oct 30 19:45:18.486826 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:45:18.486805    5907 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29889825-gkjfd_odh-ai-gateway-infra(bba9bd45-e2ea-4652-a1bf-28245075792a)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-gkjfd" podUID="bba9bd45-e2ea-4652-a1bf-28245075792a"
Oct 30 19:45:18.500717 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:18.500700    5907 scope.go:122] "RemoveContainer" containerID="0190e60c16190a9306490619027e7e934dd3c07feffff020cdb338bf48570745"
Oct 30 19:45:18.605245 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:18.605211    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j"]
Oct 30 19:45:18.669426 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:18.669396    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889825-gkjfd"]
Oct 30 19:45:18.970162 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:18.970134    5907 patch_prober.go:29] interesting pod/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.199:15021/healthz/ready\": dial tcp 10.134.0.199:15021: connect: connection refused" start-of-body=
Oct 30 19:45:18.970270 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:18.970189    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv" podUID="57838872-c482-41ec-bda0-4a517fb81dcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.199:15021/healthz/ready\": dial tcp 10.134.0.199:15021: connect: connection refused"
Oct 30 19:45:19.280524 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:19.280491    5907 scope.go:122] "RemoveContainer" containerID="77a8ec3f9f68e41ebbd76133afe4721f9633c569306c703776b4206592e37648"
Oct 30 19:45:19.496068 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:19.496035    5907 generic.go:356] "Generic (PLEG): container finished" podID="4353f4bc-7a39-4df3-82dd-b6b58ef16b09" containerID="896a3e68cdd9948d82086172e9eeedb4f17d14039b04febaddd742b557b9f193" exitCode=60
Oct 30 19:45:19.496537 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:19.496105    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp" event={"ID":"4353f4bc-7a39-4df3-82dd-b6b58ef16b09","Type":"ContainerDied","Data":"896a3e68cdd9948d82086172e9eeedb4f17d14039b04febaddd742b557b9f193"}
Oct 30 19:45:19.496537 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:19.496166    5907 scope.go:122] "RemoveContainer" containerID="ac41ee4219efb1094c4ad9fcd632e618dc44cfb3dd705be8b8a69bc3e9d59356"
Oct 30 19:45:19.496659 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:19.496586    5907 scope.go:122] "RemoveContainer" containerID="896a3e68cdd9948d82086172e9eeedb4f17d14039b04febaddd742b557b9f193"
Oct 30 19:45:19.497068 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:45:19.497039    5907 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp_odh-ai-gateway-infra(4353f4bc-7a39-4df3-82dd-b6b58ef16b09)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp" podUID="4353f4bc-7a39-4df3-82dd-b6b58ef16b09"
Oct 30 19:45:19.499368 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:19.499346    5907 generic.go:356] "Generic (PLEG): container finished" podID="982777a9-020a-4038-a0a8-c3b0fbef7bc2" containerID="e2658c11d03525f72d47c4b212ea24b9b74c57b7562f4cbd01ec7f08c03deddf" exitCode=60
Oct 30 19:45:19.499480 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:19.499470    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9" event={"ID":"982777a9-020a-4038-a0a8-c3b0fbef7bc2","Type":"ContainerDied","Data":"e2658c11d03525f72d47c4b212ea24b9b74c57b7562f4cbd01ec7f08c03deddf"}
Oct 30 19:45:19.499845 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:19.499825    5907 scope.go:122] "RemoveContainer" containerID="e2658c11d03525f72d47c4b212ea24b9b74c57b7562f4cbd01ec7f08c03deddf"
Oct 30 19:45:19.500273 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:45:19.500252    5907 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9_odh-ai-gateway-infra(982777a9-020a-4038-a0a8-c3b0fbef7bc2)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9" podUID="982777a9-020a-4038-a0a8-c3b0fbef7bc2"
Oct 30 19:45:19.548246 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:19.546494    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5" event={"ID":"a5607fe2-1ded-4e25-8361-baa33c565d00","Type":"ContainerStarted","Data":"0949893dcf3d0d5295757f78d2f2f9467c9c36c87d21aa782ef617cf38d147e1"}
Oct 30 19:45:19.552587 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:19.552565    5907 scope.go:122] "RemoveContainer" containerID="2ae6ef3ab01b7899e1592b8b75d4c1e521faae45705642dc115c981b75c940cd"
Oct 30 19:45:19.553224 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:19.553191    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn" event={"ID":"2dc2448a-45e5-4f3c-9458-ad45df73bf67","Type":"ContainerStarted","Data":"a28ed7d669a2ef9ca21971ac039180044525925c231753ec764313cd068c8edb"}
Oct 30 19:45:19.568317 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:19.568287    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp"]
Oct 30 19:45:19.680542 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:19.680502    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9"]
Oct 30 19:45:19.794144 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:19.794107    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-gkjfd"
Oct 30 19:45:19.845643 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:19.845559    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bba9bd45-e2ea-4652-a1bf-28245075792a-kube-api-access-64644\" (UniqueName: \"kubernetes.io/projected/bba9bd45-e2ea-4652-a1bf-28245075792a-kube-api-access-64644\") pod \"bba9bd45-e2ea-4652-a1bf-28245075792a\" (UID: \"bba9bd45-e2ea-4652-a1bf-28245075792a\") "
Oct 30 19:45:19.845800 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:19.845738    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bba9bd45-e2ea-4652-a1bf-28245075792a-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bba9bd45-e2ea-4652-a1bf-28245075792a-maas-api-service-ca\") pod \"bba9bd45-e2ea-4652-a1bf-28245075792a\" (UID: \"bba9bd45-e2ea-4652-a1bf-28245075792a\") "
Oct 30 19:45:19.847117 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:19.847085    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bba9bd45-e2ea-4652-a1bf-28245075792a-maas-api-service-ca" pod "bba9bd45-e2ea-4652-a1bf-28245075792a" (UID: "bba9bd45-e2ea-4652-a1bf-28245075792a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:19.849404 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:19.849382    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bba9bd45-e2ea-4652-a1bf-28245075792a-kube-api-access-64644" pod "bba9bd45-e2ea-4652-a1bf-28245075792a" (UID: "bba9bd45-e2ea-4652-a1bf-28245075792a"). InnerVolumeSpecName "kube-api-access-64644". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:19.885478 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:19.885449    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j"
Oct 30 19:45:19.947363 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:19.947212    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c8186fe3-60be-4434-aff7-e8251491d6c9-kube-api-access-6vdsg\" (UniqueName: \"kubernetes.io/projected/c8186fe3-60be-4434-aff7-e8251491d6c9-kube-api-access-6vdsg\") pod \"c8186fe3-60be-4434-aff7-e8251491d6c9\" (UID: \"c8186fe3-60be-4434-aff7-e8251491d6c9\") "
Oct 30 19:45:19.947562 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:19.947544    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c8186fe3-60be-4434-aff7-e8251491d6c9-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8186fe3-60be-4434-aff7-e8251491d6c9-maas-api-service-ca\") pod \"c8186fe3-60be-4434-aff7-e8251491d6c9\" (UID: \"c8186fe3-60be-4434-aff7-e8251491d6c9\") "
Oct 30 19:45:19.947921 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:19.947901    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8186fe3-60be-4434-aff7-e8251491d6c9-maas-api-service-ca" pod "c8186fe3-60be-4434-aff7-e8251491d6c9" (UID: "c8186fe3-60be-4434-aff7-e8251491d6c9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:19.948149 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:19.948131    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8186fe3-60be-4434-aff7-e8251491d6c9-maas-api-service-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:19.948218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:19.948156    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64644\" (UniqueName: \"kubernetes.io/projected/bba9bd45-e2ea-4652-a1bf-28245075792a-kube-api-access-64644\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:19.948218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:19.948170    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bba9bd45-e2ea-4652-a1bf-28245075792a-maas-api-service-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:19.951138 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:19.951118    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8186fe3-60be-4434-aff7-e8251491d6c9-kube-api-access-6vdsg" pod "c8186fe3-60be-4434-aff7-e8251491d6c9" (UID: "c8186fe3-60be-4434-aff7-e8251491d6c9"). InnerVolumeSpecName "kube-api-access-6vdsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:19.970258 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:19.970228    5907 patch_prober.go:29] interesting pod/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.199:15021/healthz/ready\": dial tcp 10.134.0.199:15021: connect: connection refused" start-of-body=
Oct 30 19:45:19.970357 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:19.970282    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv" podUID="57838872-c482-41ec-bda0-4a517fb81dcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.199:15021/healthz/ready\": dial tcp 10.134.0.199:15021: connect: connection refused"
Oct 30 19:45:20.050714 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.050679    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vdsg\" (UniqueName: \"kubernetes.io/projected/c8186fe3-60be-4434-aff7-e8251491d6c9-kube-api-access-6vdsg\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:20.207440 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.207131    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-67953efd-b8797cccd-pr7bp"
Oct 30 19:45:20.565455 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.565425    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j" event={"ID":"c8186fe3-60be-4434-aff7-e8251491d6c9","Type":"ContainerDied","Data":"d444eb21ae21da010d0c8ae56a48c1abd49ca0de5840310360b6d2646778f76b"}
Oct 30 19:45:20.565970 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.565480    5907 scope.go:122] "RemoveContainer" containerID="f4554c762f52b80a1784aaa2c61e701db01035cae31cb293872a764ffda15af6"
Oct 30 19:45:20.565970 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.565580    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j"
Oct 30 19:45:20.568206 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.568028    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-gkjfd"
Oct 30 19:45:20.568615 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.568595    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-gkjfd" event={"ID":"bba9bd45-e2ea-4652-a1bf-28245075792a","Type":"ContainerDied","Data":"e3b6f18809fa126e9a0dfaf16579312e5bb2e4d4a78c0620b7d2f5780cc284da"}
Oct 30 19:45:20.571251 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.571227    5907 generic.go:356] "Generic (PLEG): container finished" podID="a5607fe2-1ded-4e25-8361-baa33c565d00" containerID="0949893dcf3d0d5295757f78d2f2f9467c9c36c87d21aa782ef617cf38d147e1" exitCode=60
Oct 30 19:45:20.571364 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.571302    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5" event={"ID":"a5607fe2-1ded-4e25-8361-baa33c565d00","Type":"ContainerDied","Data":"0949893dcf3d0d5295757f78d2f2f9467c9c36c87d21aa782ef617cf38d147e1"}
Oct 30 19:45:20.571796 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.571776    5907 scope.go:122] "RemoveContainer" containerID="0949893dcf3d0d5295757f78d2f2f9467c9c36c87d21aa782ef617cf38d147e1"
Oct 30 19:45:20.572161 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:45:20.572134    5907 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5_odh-ai-gateway-infra(a5607fe2-1ded-4e25-8361-baa33c565d00)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5" podUID="a5607fe2-1ded-4e25-8361-baa33c565d00"
Oct 30 19:45:20.574389 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.573588    5907 generic.go:356] "Generic (PLEG): container finished" podID="2dc2448a-45e5-4f3c-9458-ad45df73bf67" containerID="a28ed7d669a2ef9ca21971ac039180044525925c231753ec764313cd068c8edb" exitCode=60
Oct 30 19:45:20.574389 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.573634    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn" event={"ID":"2dc2448a-45e5-4f3c-9458-ad45df73bf67","Type":"ContainerDied","Data":"a28ed7d669a2ef9ca21971ac039180044525925c231753ec764313cd068c8edb"}
Oct 30 19:45:20.574389 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.574071    5907 scope.go:122] "RemoveContainer" containerID="a28ed7d669a2ef9ca21971ac039180044525925c231753ec764313cd068c8edb"
Oct 30 19:45:20.574389 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:45:20.574345    5907 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn_odh-ai-gateway-infra(2dc2448a-45e5-4f3c-9458-ad45df73bf67)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn" podUID="2dc2448a-45e5-4f3c-9458-ad45df73bf67"
Oct 30 19:45:20.583133 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.583035    5907 scope.go:122] "RemoveContainer" containerID="b6f6830faccdf8cf3ff609e71480e25261df25efbff6a8255c7fd6f47c2c9aa6"
Oct 30 19:45:20.587066 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.587041    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw" event={"ID":"5dbb34bb-6526-4dc5-bf66-9850efbdca88","Type":"ContainerStarted","Data":"b5bef94b5a34899ccba4a3df58560204ff6946cce65bd96ad0120729aee8e721"}
Oct 30 19:45:20.605378 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.604202    5907 scope.go:122] "RemoveContainer" containerID="7b5ff0aee7629e7449a3fd87814739a001283b0254e8ec9fcb543efb79e87252"
Oct 30 19:45:20.753735 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.753708    5907 scope.go:122] "RemoveContainer" containerID="cad5bf566530385cd5073a63a62faf72347760ca81352113d10606241443ee4f"
Oct 30 19:45:20.859314 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.859232    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5"]
Oct 30 19:45:20.863278 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.863253    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9"
Oct 30 19:45:20.898929 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.898538    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j"]
Oct 30 19:45:20.933496 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.933458    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-ef9a5a-29889825-rlg5j"]
Oct 30 19:45:20.943287 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.941640    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp"
Oct 30 19:45:20.946357 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.946335    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn"]
Oct 30 19:45:20.966339 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.966314    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/982777a9-020a-4038-a0a8-c3b0fbef7bc2-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/982777a9-020a-4038-a0a8-c3b0fbef7bc2-maas-api-service-ca\") pod \"982777a9-020a-4038-a0a8-c3b0fbef7bc2\" (UID: \"982777a9-020a-4038-a0a8-c3b0fbef7bc2\") "
Oct 30 19:45:20.966474 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.966400    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4353f4bc-7a39-4df3-82dd-b6b58ef16b09-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4353f4bc-7a39-4df3-82dd-b6b58ef16b09-maas-api-service-ca\") pod \"4353f4bc-7a39-4df3-82dd-b6b58ef16b09\" (UID: \"4353f4bc-7a39-4df3-82dd-b6b58ef16b09\") "
Oct 30 19:45:20.966592 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.966573    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/982777a9-020a-4038-a0a8-c3b0fbef7bc2-kube-api-access-k868c\" (UniqueName: \"kubernetes.io/projected/982777a9-020a-4038-a0a8-c3b0fbef7bc2-kube-api-access-k868c\") pod \"982777a9-020a-4038-a0a8-c3b0fbef7bc2\" (UID: \"982777a9-020a-4038-a0a8-c3b0fbef7bc2\") "
Oct 30 19:45:20.966719 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.966630    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4353f4bc-7a39-4df3-82dd-b6b58ef16b09-kube-api-access-s5sj6\" (UniqueName: \"kubernetes.io/projected/4353f4bc-7a39-4df3-82dd-b6b58ef16b09-kube-api-access-s5sj6\") pod \"4353f4bc-7a39-4df3-82dd-b6b58ef16b09\" (UID: \"4353f4bc-7a39-4df3-82dd-b6b58ef16b09\") "
Oct 30 19:45:20.966719 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.966690    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/982777a9-020a-4038-a0a8-c3b0fbef7bc2-maas-api-service-ca" pod "982777a9-020a-4038-a0a8-c3b0fbef7bc2" (UID: "982777a9-020a-4038-a0a8-c3b0fbef7bc2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:20.966832 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.966777    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4353f4bc-7a39-4df3-82dd-b6b58ef16b09-maas-api-service-ca" pod "4353f4bc-7a39-4df3-82dd-b6b58ef16b09" (UID: "4353f4bc-7a39-4df3-82dd-b6b58ef16b09"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:20.967257 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.967230    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/982777a9-020a-4038-a0a8-c3b0fbef7bc2-maas-api-service-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:20.967350 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.967260    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4353f4bc-7a39-4df3-82dd-b6b58ef16b09-maas-api-service-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:20.968975 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.968953    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4353f4bc-7a39-4df3-82dd-b6b58ef16b09-kube-api-access-s5sj6" pod "4353f4bc-7a39-4df3-82dd-b6b58ef16b09" (UID: "4353f4bc-7a39-4df3-82dd-b6b58ef16b09"). InnerVolumeSpecName "kube-api-access-s5sj6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:20.969034 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.968996    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/982777a9-020a-4038-a0a8-c3b0fbef7bc2-kube-api-access-k868c" pod "982777a9-020a-4038-a0a8-c3b0fbef7bc2" (UID: "982777a9-020a-4038-a0a8-c3b0fbef7bc2"). InnerVolumeSpecName "kube-api-access-k868c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:20.969344 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.969324    5907 patch_prober.go:29] interesting pod/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.199:15021/healthz/ready\": dial tcp 10.134.0.199:15021: connect: connection refused" start-of-body=
Oct 30 19:45:20.969424 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.969375    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv" podUID="57838872-c482-41ec-bda0-4a517fb81dcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.199:15021/healthz/ready\": dial tcp 10.134.0.199:15021: connect: connection refused"
Oct 30 19:45:20.975758 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.975736    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889825-gkjfd"]
Oct 30 19:45:20.983725 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:20.983704    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889825-gkjfd"]
Oct 30 19:45:21.068818 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:21.068787    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5sj6\" (UniqueName: \"kubernetes.io/projected/4353f4bc-7a39-4df3-82dd-b6b58ef16b09-kube-api-access-s5sj6\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:21.068818 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:21.068818    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k868c\" (UniqueName: \"kubernetes.io/projected/982777a9-020a-4038-a0a8-c3b0fbef7bc2-kube-api-access-k868c\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:21.337887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:21.337070    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-mhv64"
Oct 30 19:45:21.597005 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:21.596932    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9"
Oct 30 19:45:21.597531 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:21.596952    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9" event={"ID":"982777a9-020a-4038-a0a8-c3b0fbef7bc2","Type":"ContainerDied","Data":"1b4d5332c1bfd9b86a98053e48e403570120d592dd09c60f5b4af625161c17c0"}
Oct 30 19:45:21.597664 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:21.597576    5907 scope.go:122] "RemoveContainer" containerID="e2658c11d03525f72d47c4b212ea24b9b74c57b7562f4cbd01ec7f08c03deddf"
Oct 30 19:45:21.607066 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:21.607040    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp"
Oct 30 19:45:21.609308 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:21.609281    5907 generic.go:356] "Generic (PLEG): container finished" podID="5dbb34bb-6526-4dc5-bf66-9850efbdca88" containerID="b5bef94b5a34899ccba4a3df58560204ff6946cce65bd96ad0120729aee8e721" exitCode=60
Oct 30 19:45:21.640884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:21.607048    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp" event={"ID":"4353f4bc-7a39-4df3-82dd-b6b58ef16b09","Type":"ContainerDied","Data":"8f654f0715d16e2280e89019de1ab7ec208f19a2e8f9f6b66f7ff80efa835e6a"}
Oct 30 19:45:21.640884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:21.638117    5907 scope.go:122] "RemoveContainer" containerID="896a3e68cdd9948d82086172e9eeedb4f17d14039b04febaddd742b557b9f193"
Oct 30 19:45:21.640884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:21.639816    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw" event={"ID":"5dbb34bb-6526-4dc5-bf66-9850efbdca88","Type":"ContainerDied","Data":"b5bef94b5a34899ccba4a3df58560204ff6946cce65bd96ad0120729aee8e721"}
Oct 30 19:45:21.640884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:21.640809    5907 scope.go:122] "RemoveContainer" containerID="b5bef94b5a34899ccba4a3df58560204ff6946cce65bd96ad0120729aee8e721"
Oct 30 19:45:21.641501 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:45:21.641456    5907 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw_odh-ai-gateway-infra(5dbb34bb-6526-4dc5-bf66-9850efbdca88)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw" podUID="5dbb34bb-6526-4dc5-bf66-9850efbdca88"
Oct 30 19:45:21.659121 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:21.658117    5907 scope.go:122] "RemoveContainer" containerID="77a8ec3f9f68e41ebbd76133afe4721f9633c569306c703776b4206592e37648"
Oct 30 19:45:21.689872 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:21.689822    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw"]
Oct 30 19:45:21.850366 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:21.850328    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9"]
Oct 30 19:45:21.856719 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:21.856691    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-fe4ec3-29889825-cqpl9"]
Oct 30 19:45:21.885810 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:21.885777    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn"
Oct 30 19:45:21.943559 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:21.943521    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp"]
Oct 30 19:45:21.944514 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:21.944436    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5"
Oct 30 19:45:21.947727 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:21.947705    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f913c6-29889825-qnbxp"]
Oct 30 19:45:21.970243 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:21.970215    5907 patch_prober.go:29] interesting pod/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.199:15021/healthz/ready\": dial tcp 10.134.0.199:15021: connect: connection refused" start-of-body=
Oct 30 19:45:21.970351 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:21.970262    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv" podUID="57838872-c482-41ec-bda0-4a517fb81dcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.199:15021/healthz/ready\": dial tcp 10.134.0.199:15021: connect: connection refused"
Oct 30 19:45:21.986116 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:21.986089    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5607fe2-1ded-4e25-8361-baa33c565d00-kube-api-access-fs8wv\" (UniqueName: \"kubernetes.io/projected/a5607fe2-1ded-4e25-8361-baa33c565d00-kube-api-access-fs8wv\") pod \"a5607fe2-1ded-4e25-8361-baa33c565d00\" (UID: \"a5607fe2-1ded-4e25-8361-baa33c565d00\") "
Oct 30 19:45:21.986229 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:21.986216    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a5607fe2-1ded-4e25-8361-baa33c565d00-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5607fe2-1ded-4e25-8361-baa33c565d00-maas-api-service-ca\") pod \"a5607fe2-1ded-4e25-8361-baa33c565d00\" (UID: \"a5607fe2-1ded-4e25-8361-baa33c565d00\") "
Oct 30 19:45:21.986320 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:21.986308    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2dc2448a-45e5-4f3c-9458-ad45df73bf67-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dc2448a-45e5-4f3c-9458-ad45df73bf67-maas-api-service-ca\") pod \"2dc2448a-45e5-4f3c-9458-ad45df73bf67\" (UID: \"2dc2448a-45e5-4f3c-9458-ad45df73bf67\") "
Oct 30 19:45:21.986450 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:21.986439    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2dc2448a-45e5-4f3c-9458-ad45df73bf67-kube-api-access-jfvf2\" (UniqueName: \"kubernetes.io/projected/2dc2448a-45e5-4f3c-9458-ad45df73bf67-kube-api-access-jfvf2\") pod \"2dc2448a-45e5-4f3c-9458-ad45df73bf67\" (UID: \"2dc2448a-45e5-4f3c-9458-ad45df73bf67\") "
Oct 30 19:45:21.989130 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:21.989100    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2dc2448a-45e5-4f3c-9458-ad45df73bf67-maas-api-service-ca" pod "2dc2448a-45e5-4f3c-9458-ad45df73bf67" (UID: "2dc2448a-45e5-4f3c-9458-ad45df73bf67"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:21.989746 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:21.989706    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5607fe2-1ded-4e25-8361-baa33c565d00-maas-api-service-ca" pod "a5607fe2-1ded-4e25-8361-baa33c565d00" (UID: "a5607fe2-1ded-4e25-8361-baa33c565d00"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:21.989852 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:21.989800    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5607fe2-1ded-4e25-8361-baa33c565d00-kube-api-access-fs8wv" pod "a5607fe2-1ded-4e25-8361-baa33c565d00" (UID: "a5607fe2-1ded-4e25-8361-baa33c565d00"). InnerVolumeSpecName "kube-api-access-fs8wv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:21.990922 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:21.990896    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fs8wv\" (UniqueName: \"kubernetes.io/projected/a5607fe2-1ded-4e25-8361-baa33c565d00-kube-api-access-fs8wv\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:21.991072 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:21.990928    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5607fe2-1ded-4e25-8361-baa33c565d00-maas-api-service-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:21.991072 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:21.990964    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dc2448a-45e5-4f3c-9458-ad45df73bf67-maas-api-service-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:22.007360 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:22.007326    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2dc2448a-45e5-4f3c-9458-ad45df73bf67-kube-api-access-jfvf2" pod "2dc2448a-45e5-4f3c-9458-ad45df73bf67" (UID: "2dc2448a-45e5-4f3c-9458-ad45df73bf67"). InnerVolumeSpecName "kube-api-access-jfvf2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:22.092718 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:22.092659    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jfvf2\" (UniqueName: \"kubernetes.io/projected/2dc2448a-45e5-4f3c-9458-ad45df73bf67-kube-api-access-jfvf2\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:22.340800 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:22.340720    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4353f4bc-7a39-4df3-82dd-b6b58ef16b09" path="/var/lib/kubelet/pods/4353f4bc-7a39-4df3-82dd-b6b58ef16b09/volumes"
Oct 30 19:45:22.341466 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:22.341422    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="982777a9-020a-4038-a0a8-c3b0fbef7bc2" path="/var/lib/kubelet/pods/982777a9-020a-4038-a0a8-c3b0fbef7bc2/volumes"
Oct 30 19:45:22.341848 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:22.341832    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bba9bd45-e2ea-4652-a1bf-28245075792a" path="/var/lib/kubelet/pods/bba9bd45-e2ea-4652-a1bf-28245075792a/volumes"
Oct 30 19:45:22.342589 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:22.342573    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c8186fe3-60be-4434-aff7-e8251491d6c9" path="/var/lib/kubelet/pods/c8186fe3-60be-4434-aff7-e8251491d6c9/volumes"
Oct 30 19:45:22.624348 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:22.624321    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5"
Oct 30 19:45:22.624348 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:22.624328    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5" event={"ID":"a5607fe2-1ded-4e25-8361-baa33c565d00","Type":"ContainerDied","Data":"bb1b11ccf2f8bbbe02c2c602ecbc4c6de8b37182dbf24d23f1231c0242fed979"}
Oct 30 19:45:22.624962 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:22.624401    5907 scope.go:122] "RemoveContainer" containerID="0949893dcf3d0d5295757f78d2f2f9467c9c36c87d21aa782ef617cf38d147e1"
Oct 30 19:45:22.626257 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:22.626235    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn"
Oct 30 19:45:22.626748 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:22.626722    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn" event={"ID":"2dc2448a-45e5-4f3c-9458-ad45df73bf67","Type":"ContainerDied","Data":"c70acc65bb47f13ed654d6dad367a8f6f25c6d249c209f8e53a7b7267ab9ce60"}
Oct 30 19:45:22.639405 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:22.639383    5907 scope.go:122] "RemoveContainer" containerID="a28ed7d669a2ef9ca21971ac039180044525925c231753ec764313cd068c8edb"
Oct 30 19:45:22.762236 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:22.762196    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5"]
Oct 30 19:45:22.776479 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:22.776447    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d797d2-29889825-k4nt5"]
Oct 30 19:45:22.818939 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:22.818907    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn"]
Oct 30 19:45:22.823805 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:22.823780    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w5-e9de60-29889825-4k6cn"]
Oct 30 19:45:22.824428 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:22.824408    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw"
Oct 30 19:45:22.906109 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:22.906006    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5dbb34bb-6526-4dc5-bf66-9850efbdca88-kube-api-access-25lvf\" (UniqueName: \"kubernetes.io/projected/5dbb34bb-6526-4dc5-bf66-9850efbdca88-kube-api-access-25lvf\") pod \"5dbb34bb-6526-4dc5-bf66-9850efbdca88\" (UID: \"5dbb34bb-6526-4dc5-bf66-9850efbdca88\") "
Oct 30 19:45:22.906293 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:22.906275    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5dbb34bb-6526-4dc5-bf66-9850efbdca88-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dbb34bb-6526-4dc5-bf66-9850efbdca88-maas-api-service-ca\") pod \"5dbb34bb-6526-4dc5-bf66-9850efbdca88\" (UID: \"5dbb34bb-6526-4dc5-bf66-9850efbdca88\") "
Oct 30 19:45:22.906648 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:22.906624    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dbb34bb-6526-4dc5-bf66-9850efbdca88-maas-api-service-ca" pod "5dbb34bb-6526-4dc5-bf66-9850efbdca88" (UID: "5dbb34bb-6526-4dc5-bf66-9850efbdca88"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:22.906966 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:22.906948    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dbb34bb-6526-4dc5-bf66-9850efbdca88-maas-api-service-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:22.908329 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:22.908310    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dbb34bb-6526-4dc5-bf66-9850efbdca88-kube-api-access-25lvf" pod "5dbb34bb-6526-4dc5-bf66-9850efbdca88" (UID: "5dbb34bb-6526-4dc5-bf66-9850efbdca88"). InnerVolumeSpecName "kube-api-access-25lvf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:22.974317 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:22.974292    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:45:22.974816 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:22.974795    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:45:22.975635 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:22.975615    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:45:23.008045 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.008017    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25lvf\" (UniqueName: \"kubernetes.io/projected/5dbb34bb-6526-4dc5-bf66-9850efbdca88-kube-api-access-25lvf\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:23.311851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.311757    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-0c305f-596bs"]
Oct 30 19:45:23.312562 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312543    5907 cpu_manager.go:414] "containerMap: removing container" podUID="a5607fe2-1ded-4e25-8361-baa33c565d00" containerName="cleanup"
Oct 30 19:45:23.312562 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312559    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5607fe2-1ded-4e25-8361-baa33c565d00" containerName="cleanup"
Oct 30 19:45:23.312700 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312572    5907 cpu_manager.go:414] "containerMap: removing container" podUID="4353f4bc-7a39-4df3-82dd-b6b58ef16b09" containerName="cleanup"
Oct 30 19:45:23.312700 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312578    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4353f4bc-7a39-4df3-82dd-b6b58ef16b09" containerName="cleanup"
Oct 30 19:45:23.312700 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312590    5907 cpu_manager.go:414] "containerMap: removing container" podUID="c8186fe3-60be-4434-aff7-e8251491d6c9" containerName="cleanup"
Oct 30 19:45:23.312700 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312595    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c8186fe3-60be-4434-aff7-e8251491d6c9" containerName="cleanup"
Oct 30 19:45:23.312700 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312605    5907 cpu_manager.go:414] "containerMap: removing container" podUID="5dbb34bb-6526-4dc5-bf66-9850efbdca88" containerName="cleanup"
Oct 30 19:45:23.312700 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312609    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5dbb34bb-6526-4dc5-bf66-9850efbdca88" containerName="cleanup"
Oct 30 19:45:23.312700 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312622    5907 cpu_manager.go:414] "containerMap: removing container" podUID="a5607fe2-1ded-4e25-8361-baa33c565d00" containerName="cleanup"
Oct 30 19:45:23.312700 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312627    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5607fe2-1ded-4e25-8361-baa33c565d00" containerName="cleanup"
Oct 30 19:45:23.312700 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312634    5907 cpu_manager.go:414] "containerMap: removing container" podUID="bba9bd45-e2ea-4652-a1bf-28245075792a" containerName="cleanup"
Oct 30 19:45:23.312700 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312638    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bba9bd45-e2ea-4652-a1bf-28245075792a" containerName="cleanup"
Oct 30 19:45:23.312700 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312663    5907 cpu_manager.go:414] "containerMap: removing container" podUID="4353f4bc-7a39-4df3-82dd-b6b58ef16b09" containerName="cleanup"
Oct 30 19:45:23.312700 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312667    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4353f4bc-7a39-4df3-82dd-b6b58ef16b09" containerName="cleanup"
Oct 30 19:45:23.312700 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312675    5907 cpu_manager.go:414] "containerMap: removing container" podUID="982777a9-020a-4038-a0a8-c3b0fbef7bc2" containerName="cleanup"
Oct 30 19:45:23.312700 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312679    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="982777a9-020a-4038-a0a8-c3b0fbef7bc2" containerName="cleanup"
Oct 30 19:45:23.312700 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312696    5907 cpu_manager.go:414] "containerMap: removing container" podUID="982777a9-020a-4038-a0a8-c3b0fbef7bc2" containerName="cleanup"
Oct 30 19:45:23.312700 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312700    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="982777a9-020a-4038-a0a8-c3b0fbef7bc2" containerName="cleanup"
Oct 30 19:45:23.312700 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312708    5907 cpu_manager.go:414] "containerMap: removing container" podUID="5dbb34bb-6526-4dc5-bf66-9850efbdca88" containerName="cleanup"
Oct 30 19:45:23.313239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312712    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5dbb34bb-6526-4dc5-bf66-9850efbdca88" containerName="cleanup"
Oct 30 19:45:23.313239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312719    5907 cpu_manager.go:414] "containerMap: removing container" podUID="2dc2448a-45e5-4f3c-9458-ad45df73bf67" containerName="cleanup"
Oct 30 19:45:23.313239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312723    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2dc2448a-45e5-4f3c-9458-ad45df73bf67" containerName="cleanup"
Oct 30 19:45:23.313239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312730    5907 cpu_manager.go:414] "containerMap: removing container" podUID="982777a9-020a-4038-a0a8-c3b0fbef7bc2" containerName="cleanup"
Oct 30 19:45:23.313239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312734    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="982777a9-020a-4038-a0a8-c3b0fbef7bc2" containerName="cleanup"
Oct 30 19:45:23.313239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312741    5907 cpu_manager.go:414] "containerMap: removing container" podUID="2dc2448a-45e5-4f3c-9458-ad45df73bf67" containerName="cleanup"
Oct 30 19:45:23.313239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312745    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2dc2448a-45e5-4f3c-9458-ad45df73bf67" containerName="cleanup"
Oct 30 19:45:23.313239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312751    5907 cpu_manager.go:414] "containerMap: removing container" podUID="5dbb34bb-6526-4dc5-bf66-9850efbdca88" containerName="cleanup"
Oct 30 19:45:23.313239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312755    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5dbb34bb-6526-4dc5-bf66-9850efbdca88" containerName="cleanup"
Oct 30 19:45:23.313239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312764    5907 cpu_manager.go:414] "containerMap: removing container" podUID="bba9bd45-e2ea-4652-a1bf-28245075792a" containerName="cleanup"
Oct 30 19:45:23.313239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312769    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bba9bd45-e2ea-4652-a1bf-28245075792a" containerName="cleanup"
Oct 30 19:45:23.313239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312774    5907 cpu_manager.go:414] "containerMap: removing container" podUID="c8186fe3-60be-4434-aff7-e8251491d6c9" containerName="cleanup"
Oct 30 19:45:23.313239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312778    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c8186fe3-60be-4434-aff7-e8251491d6c9" containerName="cleanup"
Oct 30 19:45:23.313239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312896    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="bba9bd45-e2ea-4652-a1bf-28245075792a" containerName="cleanup"
Oct 30 19:45:23.313239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312906    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="4353f4bc-7a39-4df3-82dd-b6b58ef16b09" containerName="cleanup"
Oct 30 19:45:23.313239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312911    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="4353f4bc-7a39-4df3-82dd-b6b58ef16b09" containerName="cleanup"
Oct 30 19:45:23.313239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312917    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="982777a9-020a-4038-a0a8-c3b0fbef7bc2" containerName="cleanup"
Oct 30 19:45:23.313239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312923    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="c8186fe3-60be-4434-aff7-e8251491d6c9" containerName="cleanup"
Oct 30 19:45:23.313239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312932    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5607fe2-1ded-4e25-8361-baa33c565d00" containerName="cleanup"
Oct 30 19:45:23.313239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312938    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="982777a9-020a-4038-a0a8-c3b0fbef7bc2" containerName="cleanup"
Oct 30 19:45:23.313239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312943    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="5dbb34bb-6526-4dc5-bf66-9850efbdca88" containerName="cleanup"
Oct 30 19:45:23.313239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312948    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="2dc2448a-45e5-4f3c-9458-ad45df73bf67" containerName="cleanup"
Oct 30 19:45:23.313239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312955    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="982777a9-020a-4038-a0a8-c3b0fbef7bc2" containerName="cleanup"
Oct 30 19:45:23.313239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312962    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="2dc2448a-45e5-4f3c-9458-ad45df73bf67" containerName="cleanup"
Oct 30 19:45:23.313239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312969    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="bba9bd45-e2ea-4652-a1bf-28245075792a" containerName="cleanup"
Oct 30 19:45:23.313239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312975    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="2dc2448a-45e5-4f3c-9458-ad45df73bf67" containerName="cleanup"
Oct 30 19:45:23.313239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312980    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5607fe2-1ded-4e25-8361-baa33c565d00" containerName="cleanup"
Oct 30 19:45:23.313239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312987    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="c8186fe3-60be-4434-aff7-e8251491d6c9" containerName="cleanup"
Oct 30 19:45:23.313239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312992    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="5dbb34bb-6526-4dc5-bf66-9850efbdca88" containerName="cleanup"
Oct 30 19:45:23.313239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.312998    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5607fe2-1ded-4e25-8361-baa33c565d00" containerName="cleanup"
Oct 30 19:45:23.317007 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.316989    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-0c305f-596bs"
Oct 30 19:45:23.322999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.322974    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f7f11c63-7327-488a-8749-d7f6c395de05-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w0-0c305f-596bs\" (UID: \"f7f11c63-7327-488a-8749-d7f6c395de05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-0c305f-596bs"
Oct 30 19:45:23.323144 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.323095    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f7f11c63-7327-488a-8749-d7f6c395de05-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w0-0c305f-596bs\" (UID: \"f7f11c63-7327-488a-8749-d7f6c395de05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-0c305f-596bs"
Oct 30 19:45:23.323369 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.323340    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqhlq\" (UniqueName: \"kubernetes.io/projected/f7f11c63-7327-488a-8749-d7f6c395de05-kube-api-access-hqhlq\") pod \"maas-api-revoke-keys-e2e-shared-a-w0-0c305f-596bs\" (UID: \"f7f11c63-7327-488a-8749-d7f6c395de05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-0c305f-596bs"
Oct 30 19:45:23.345482 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.345447    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-0c305f-596bs"]
Oct 30 19:45:23.392756 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.392721    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:45:23.424769 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.424740    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f7f11c63-7327-488a-8749-d7f6c395de05-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w0-0c305f-596bs\" (UID: \"f7f11c63-7327-488a-8749-d7f6c395de05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-0c305f-596bs"
Oct 30 19:45:23.424945 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.424819    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f7f11c63-7327-488a-8749-d7f6c395de05-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w0-0c305f-596bs\" (UID: \"f7f11c63-7327-488a-8749-d7f6c395de05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-0c305f-596bs"
Oct 30 19:45:23.424997 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.424955    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqhlq\" (UniqueName: \"kubernetes.io/projected/f7f11c63-7327-488a-8749-d7f6c395de05-kube-api-access-hqhlq\") pod \"maas-api-revoke-keys-e2e-shared-a-w0-0c305f-596bs\" (UID: \"f7f11c63-7327-488a-8749-d7f6c395de05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-0c305f-596bs"
Oct 30 19:45:23.425581 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.425556    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f7f11c63-7327-488a-8749-d7f6c395de05-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w0-0c305f-596bs\" (UID: \"f7f11c63-7327-488a-8749-d7f6c395de05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-0c305f-596bs"
Oct 30 19:45:23.427289 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.427262    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f7f11c63-7327-488a-8749-d7f6c395de05-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w0-0c305f-596bs\" (UID: \"f7f11c63-7327-488a-8749-d7f6c395de05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-0c305f-596bs"
Oct 30 19:45:23.455927 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.455906    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqhlq\" (UniqueName: \"kubernetes.io/projected/f7f11c63-7327-488a-8749-d7f6c395de05-kube-api-access-hqhlq\") pod \"maas-api-revoke-keys-e2e-shared-a-w0-0c305f-596bs\" (UID: \"f7f11c63-7327-488a-8749-d7f6c395de05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-0c305f-596bs"
Oct 30 19:45:23.631940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.631900    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-0c305f-596bs"
Oct 30 19:45:23.647653 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.647623    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw"
Oct 30 19:45:23.647806 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.647670    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw" event={"ID":"5dbb34bb-6526-4dc5-bf66-9850efbdca88","Type":"ContainerDied","Data":"c67b661ba78475d2564397c024e29cb9771aa4f7e1bb7f2200e9a125341fd626"}
Oct 30 19:45:23.647806 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.647736    5907 scope.go:122] "RemoveContainer" containerID="b5bef94b5a34899ccba4a3df58560204ff6946cce65bd96ad0120729aee8e721"
Oct 30 19:45:23.723809 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.723775    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw"]
Oct 30 19:45:23.738823 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.738302    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-291d35-29889825-ts2xw"]
Oct 30 19:45:23.957793 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:23.957761    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-0c305f-596bs"]
Oct 30 19:45:23.959046 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:45:23.959013    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7f11c63_7327_488a_8749_d7f6c395de05.slice/crio-8105ae98a72165dcefc5af0559fe4a01bd4e240676c242df359624b5a9720fad WatchSource:0}: Error finding container 8105ae98a72165dcefc5af0559fe4a01bd4e240676c242df359624b5a9720fad: Status 404 returned error can't find the container with id 8105ae98a72165dcefc5af0559fe4a01bd4e240676c242df359624b5a9720fad
Oct 30 19:45:24.290588 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:24.290553    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2dc2448a-45e5-4f3c-9458-ad45df73bf67" path="/var/lib/kubelet/pods/2dc2448a-45e5-4f3c-9458-ad45df73bf67/volumes"
Oct 30 19:45:24.291189 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:24.291166    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5dbb34bb-6526-4dc5-bf66-9850efbdca88" path="/var/lib/kubelet/pods/5dbb34bb-6526-4dc5-bf66-9850efbdca88/volumes"
Oct 30 19:45:24.291660 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:24.291640    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a5607fe2-1ded-4e25-8361-baa33c565d00" path="/var/lib/kubelet/pods/a5607fe2-1ded-4e25-8361-baa33c565d00/volumes"
Oct 30 19:45:24.661432 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:24.661393    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-0c305f-596bs" event={"ID":"f7f11c63-7327-488a-8749-d7f6c395de05","Type":"ContainerStarted","Data":"aff22d4616dc59538c5919e28607117b6aaef0f6f3fee9a5b728a9dad011a7ad"}
Oct 30 19:45:24.661917 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:24.661487    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-0c305f-596bs" event={"ID":"f7f11c63-7327-488a-8749-d7f6c395de05","Type":"ContainerStarted","Data":"8105ae98a72165dcefc5af0559fe4a01bd4e240676c242df359624b5a9720fad"}
Oct 30 19:45:24.712455 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:24.712403    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-0c305f-596bs" podStartSLOduration=1.712389495 podStartE2EDuration="1.712389495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:24.707026525 +0000 UTC m=+1436.549958465" watchObservedRunningTime="2026-10-30 19:45:24.712389495 +0000 UTC m=+1436.555321433"
Oct 30 19:45:25.676799 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:25.676761    5907 generic.go:356] "Generic (PLEG): container finished" podID="f7f11c63-7327-488a-8749-d7f6c395de05" containerID="aff22d4616dc59538c5919e28607117b6aaef0f6f3fee9a5b728a9dad011a7ad" exitCode=0
Oct 30 19:45:25.676799 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:25.676789    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-0c305f-596bs" event={"ID":"f7f11c63-7327-488a-8749-d7f6c395de05","Type":"ContainerDied","Data":"aff22d4616dc59538c5919e28607117b6aaef0f6f3fee9a5b728a9dad011a7ad"}
Oct 30 19:45:26.054970 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:26.054879    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-eb1d94ec-gnqqd"]
Oct 30 19:45:26.060978 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:26.060948    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-eb1d94ec-gnqqd"]
Oct 30 19:45:26.291389 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:26.291350    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a472ff69-84d7-4e3a-ac85-317f6f4c95aa" path="/var/lib/kubelet/pods/a472ff69-84d7-4e3a-ac85-317f6f4c95aa/volumes"
Oct 30 19:45:26.849330 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:26.849306    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-0c305f-596bs"
Oct 30 19:45:26.877953 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:26.877925    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f7f11c63-7327-488a-8749-d7f6c395de05-kube-api-access-hqhlq\" (UniqueName: \"kubernetes.io/projected/f7f11c63-7327-488a-8749-d7f6c395de05-kube-api-access-hqhlq\") pod \"f7f11c63-7327-488a-8749-d7f6c395de05\" (UID: \"f7f11c63-7327-488a-8749-d7f6c395de05\") "
Oct 30 19:45:26.878093 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:26.878037    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f7f11c63-7327-488a-8749-d7f6c395de05-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f7f11c63-7327-488a-8749-d7f6c395de05-maas-api-xks-ca\") pod \"f7f11c63-7327-488a-8749-d7f6c395de05\" (UID: \"f7f11c63-7327-488a-8749-d7f6c395de05\") "
Oct 30 19:45:26.878178 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:26.878159    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f7f11c63-7327-488a-8749-d7f6c395de05-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f7f11c63-7327-488a-8749-d7f6c395de05-maas-api-openshift-ca\") pod \"f7f11c63-7327-488a-8749-d7f6c395de05\" (UID: \"f7f11c63-7327-488a-8749-d7f6c395de05\") "
Oct 30 19:45:26.878581 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:26.878482    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7f11c63-7327-488a-8749-d7f6c395de05-maas-api-openshift-ca" pod "f7f11c63-7327-488a-8749-d7f6c395de05" (UID: "f7f11c63-7327-488a-8749-d7f6c395de05"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:26.878809 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:26.878792    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f7f11c63-7327-488a-8749-d7f6c395de05-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:26.880144 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:26.880120    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7f11c63-7327-488a-8749-d7f6c395de05-kube-api-access-hqhlq" pod "f7f11c63-7327-488a-8749-d7f6c395de05" (UID: "f7f11c63-7327-488a-8749-d7f6c395de05"). InnerVolumeSpecName "kube-api-access-hqhlq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:26.880247 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:26.880223    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7f11c63-7327-488a-8749-d7f6c395de05-maas-api-xks-ca" pod "f7f11c63-7327-488a-8749-d7f6c395de05" (UID: "f7f11c63-7327-488a-8749-d7f6c395de05"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:45:26.980805 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:26.980723    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqhlq\" (UniqueName: \"kubernetes.io/projected/f7f11c63-7327-488a-8749-d7f6c395de05-kube-api-access-hqhlq\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:26.980805 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:26.980753    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f7f11c63-7327-488a-8749-d7f6c395de05-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:27.697472 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:27.697447    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-0c305f-596bs"
Oct 30 19:45:27.697639 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:27.697479    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-0c305f-596bs" event={"ID":"f7f11c63-7327-488a-8749-d7f6c395de05","Type":"ContainerDied","Data":"8105ae98a72165dcefc5af0559fe4a01bd4e240676c242df359624b5a9720fad"}
Oct 30 19:45:27.697639 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:27.697533    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8105ae98a72165dcefc5af0559fe4a01bd4e240676c242df359624b5a9720fad"
Oct 30 19:45:28.056422 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.056345    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-67953efd-956bfdb9-4ftfp"]
Oct 30 19:45:28.057650 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.057629    5907 cpu_manager.go:414] "containerMap: removing container" podUID="f7f11c63-7327-488a-8749-d7f6c395de05" containerName="revoke-keys"
Oct 30 19:45:28.057696 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.057654    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f7f11c63-7327-488a-8749-d7f6c395de05" containerName="revoke-keys"
Oct 30 19:45:28.057696 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.057681    5907 cpu_manager.go:414] "containerMap: removing container" podUID="c8186fe3-60be-4434-aff7-e8251491d6c9" containerName="cleanup"
Oct 30 19:45:28.057696 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.057689    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c8186fe3-60be-4434-aff7-e8251491d6c9" containerName="cleanup"
Oct 30 19:45:28.057800 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.057716    5907 cpu_manager.go:414] "containerMap: removing container" podUID="a5607fe2-1ded-4e25-8361-baa33c565d00" containerName="cleanup"
Oct 30 19:45:28.057800 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.057724    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5607fe2-1ded-4e25-8361-baa33c565d00" containerName="cleanup"
Oct 30 19:45:28.057800 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.057736    5907 cpu_manager.go:414] "containerMap: removing container" podUID="bba9bd45-e2ea-4652-a1bf-28245075792a" containerName="cleanup"
Oct 30 19:45:28.057800 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.057743    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bba9bd45-e2ea-4652-a1bf-28245075792a" containerName="cleanup"
Oct 30 19:45:28.057800 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.057789    5907 cpu_manager.go:414] "containerMap: removing container" podUID="4353f4bc-7a39-4df3-82dd-b6b58ef16b09" containerName="cleanup"
Oct 30 19:45:28.057800 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.057796    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4353f4bc-7a39-4df3-82dd-b6b58ef16b09" containerName="cleanup"
Oct 30 19:45:28.058000 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.057836    5907 cpu_manager.go:414] "containerMap: removing container" podUID="2dc2448a-45e5-4f3c-9458-ad45df73bf67" containerName="cleanup"
Oct 30 19:45:28.058000 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.057843    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2dc2448a-45e5-4f3c-9458-ad45df73bf67" containerName="cleanup"
Oct 30 19:45:28.058069 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.058028    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="bba9bd45-e2ea-4652-a1bf-28245075792a" containerName="cleanup"
Oct 30 19:45:28.058069 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.058041    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="4353f4bc-7a39-4df3-82dd-b6b58ef16b09" containerName="cleanup"
Oct 30 19:45:28.058069 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.058055    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="c8186fe3-60be-4434-aff7-e8251491d6c9" containerName="cleanup"
Oct 30 19:45:28.058164 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.058071    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="f7f11c63-7327-488a-8749-d7f6c395de05" containerName="revoke-keys"
Oct 30 19:45:28.058164 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.058082    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="5dbb34bb-6526-4dc5-bf66-9850efbdca88" containerName="cleanup"
Oct 30 19:45:28.063060 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.063036    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67953efd-956bfdb9-4ftfp"
Oct 30 19:45:28.081670 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.081650    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-67953efd-serving-cert\""
Oct 30 19:45:28.097914 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.097853    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsb59\" (UniqueName: \"kubernetes.io/projected/364ccf73-2cd0-4922-9460-daca3b6f44e1-kube-api-access-tsb59\") pod \"maas-api-e2e-mt-67953efd-956bfdb9-4ftfp\" (UID: \"364ccf73-2cd0-4922-9460-daca3b6f44e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-67953efd-956bfdb9-4ftfp"
Oct 30 19:45:28.098079 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.097976    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/364ccf73-2cd0-4922-9460-daca3b6f44e1-metrics-certs\") pod \"maas-api-e2e-mt-67953efd-956bfdb9-4ftfp\" (UID: \"364ccf73-2cd0-4922-9460-daca3b6f44e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-67953efd-956bfdb9-4ftfp"
Oct 30 19:45:28.098079 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.098045    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-67953efd-956bfdb9-4ftfp"]
Oct 30 19:45:28.098206 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.098122    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/364ccf73-2cd0-4922-9460-daca3b6f44e1-maas-api-tls\") pod \"maas-api-e2e-mt-67953efd-956bfdb9-4ftfp\" (UID: \"364ccf73-2cd0-4922-9460-daca3b6f44e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-67953efd-956bfdb9-4ftfp"
Oct 30 19:45:28.098326 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.098307    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/364ccf73-2cd0-4922-9460-daca3b6f44e1-openshift-service-ca\") pod \"maas-api-e2e-mt-67953efd-956bfdb9-4ftfp\" (UID: \"364ccf73-2cd0-4922-9460-daca3b6f44e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-67953efd-956bfdb9-4ftfp"
Oct 30 19:45:28.199357 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.199328    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/364ccf73-2cd0-4922-9460-daca3b6f44e1-openshift-service-ca\") pod \"maas-api-e2e-mt-67953efd-956bfdb9-4ftfp\" (UID: \"364ccf73-2cd0-4922-9460-daca3b6f44e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-67953efd-956bfdb9-4ftfp"
Oct 30 19:45:28.199523 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.199448    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsb59\" (UniqueName: \"kubernetes.io/projected/364ccf73-2cd0-4922-9460-daca3b6f44e1-kube-api-access-tsb59\") pod \"maas-api-e2e-mt-67953efd-956bfdb9-4ftfp\" (UID: \"364ccf73-2cd0-4922-9460-daca3b6f44e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-67953efd-956bfdb9-4ftfp"
Oct 30 19:45:28.199523 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.199477    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/364ccf73-2cd0-4922-9460-daca3b6f44e1-metrics-certs\") pod \"maas-api-e2e-mt-67953efd-956bfdb9-4ftfp\" (UID: \"364ccf73-2cd0-4922-9460-daca3b6f44e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-67953efd-956bfdb9-4ftfp"
Oct 30 19:45:28.199646 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.199591    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/364ccf73-2cd0-4922-9460-daca3b6f44e1-maas-api-tls\") pod \"maas-api-e2e-mt-67953efd-956bfdb9-4ftfp\" (UID: \"364ccf73-2cd0-4922-9460-daca3b6f44e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-67953efd-956bfdb9-4ftfp"
Oct 30 19:45:28.200141 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.200115    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/364ccf73-2cd0-4922-9460-daca3b6f44e1-openshift-service-ca\") pod \"maas-api-e2e-mt-67953efd-956bfdb9-4ftfp\" (UID: \"364ccf73-2cd0-4922-9460-daca3b6f44e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-67953efd-956bfdb9-4ftfp"
Oct 30 19:45:28.202598 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.202567    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/364ccf73-2cd0-4922-9460-daca3b6f44e1-metrics-certs\") pod \"maas-api-e2e-mt-67953efd-956bfdb9-4ftfp\" (UID: \"364ccf73-2cd0-4922-9460-daca3b6f44e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-67953efd-956bfdb9-4ftfp"
Oct 30 19:45:28.202721 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.202674    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/364ccf73-2cd0-4922-9460-daca3b6f44e1-maas-api-tls\") pod \"maas-api-e2e-mt-67953efd-956bfdb9-4ftfp\" (UID: \"364ccf73-2cd0-4922-9460-daca3b6f44e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-67953efd-956bfdb9-4ftfp"
Oct 30 19:45:28.318677 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.318422    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsb59\" (UniqueName: \"kubernetes.io/projected/364ccf73-2cd0-4922-9460-daca3b6f44e1-kube-api-access-tsb59\") pod \"maas-api-e2e-mt-67953efd-956bfdb9-4ftfp\" (UID: \"364ccf73-2cd0-4922-9460-daca3b6f44e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-67953efd-956bfdb9-4ftfp"
Oct 30 19:45:28.381600 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.381569    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67953efd-956bfdb9-4ftfp"
Oct 30 19:45:28.658885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.642926    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:45:28.767888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:28.750490    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-67953efd-956bfdb9-4ftfp"]
Oct 30 19:45:28.790941 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:45:28.787016    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod364ccf73_2cd0_4922_9460_daca3b6f44e1.slice/crio-d959330f652b2831644d84a95abcd2820c381a6c85a924efc2f5843d656570b0 WatchSource:0}: Error finding container d959330f652b2831644d84a95abcd2820c381a6c85a924efc2f5843d656570b0: Status 404 returned error can't find the container with id d959330f652b2831644d84a95abcd2820c381a6c85a924efc2f5843d656570b0
Oct 30 19:45:29.739453 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:29.739420    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67953efd-956bfdb9-4ftfp" event={"ID":"364ccf73-2cd0-4922-9460-daca3b6f44e1","Type":"ContainerStarted","Data":"f4bb8b956dfa105291eaef426473484403ff188cb259a28d564513c3619069c0"}
Oct 30 19:45:29.739887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:29.739471    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67953efd-956bfdb9-4ftfp" event={"ID":"364ccf73-2cd0-4922-9460-daca3b6f44e1","Type":"ContainerStarted","Data":"d959330f652b2831644d84a95abcd2820c381a6c85a924efc2f5843d656570b0"}
Oct 30 19:45:29.739887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:29.739522    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67953efd-956bfdb9-4ftfp"
Oct 30 19:45:29.804780 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:29.804726    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67953efd-956bfdb9-4ftfp" podStartSLOduration=2.481853402 podStartE2EDuration="2.804710954s" totalImagesPullingTime="322.857552ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:29.802077122 +0000 UTC m=+1441.645009062" watchObservedRunningTime="2026-10-30 19:45:29.804710954 +0000 UTC m=+1441.647642890"
Oct 30 19:45:31.008887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:31.007732    5907 scope.go:122] "RemoveContainer" containerID="78b5f480af5e2be25256646ae63185435b206608f477a40828980aa6007f2c74"
Oct 30 19:45:31.227906 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:31.222045    5907 scope.go:122] "RemoveContainer" containerID="8ab75d85fd7dd1932fe55faa0d97064d72c19956df816c68eb0bad55fd1d399c"
Oct 30 19:45:31.531885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:31.529227    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:31.746897 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:31.746022    5907 prober.go:122] "Probe failed" probeType="Liveness" pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6" podUID="c3f73773-6c2f-47e4-a5a4-c57916f64ac9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.176:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:32.205896 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:32.201149    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:32.232538 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:32.232491    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6" podUID="c3f73773-6c2f-47e4-a5a4-c57916f64ac9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.176:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:32.361445 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:32.358783    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:32.531006 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:32.530953    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" probeResult="failure" output="Get \"https://10.134.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:32.678656 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:32.666944    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:45:32.678656 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:32.666993    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"]
Oct 30 19:45:33.137017 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:33.133993    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:33.147688 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:33.147653    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:45:33.209207 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:33.209167    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 19:45:33.220172 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:33.217484    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:45:33.341944 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:33.326949    5907 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:33.636651 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:33.636619    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:45:33.788698 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:33.788640    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-0c305f-5f5c4698b6-kzfnv"]
Oct 30 19:45:33.854195 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:33.854169    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0c305f-8677844c-mdcqk"]
Oct 30 19:45:35.465264 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:35.465215    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq"]
Oct 30 19:45:35.465810 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:35.465484    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq" podUID="d71d8254-621d-4cac-94ea-28c912d5a45b" containerName="maas-api" containerID="cri-o://20239eca8c288234216eab29e978f7e986ebef29c49cde68ed39bab6e49ef441" gracePeriod=30
Oct 30 19:45:35.600119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:35.600081    5907 generic.go:356] "Generic (PLEG): container finished" podID="d71d8254-621d-4cac-94ea-28c912d5a45b" containerID="20239eca8c288234216eab29e978f7e986ebef29c49cde68ed39bab6e49ef441" exitCode=0
Oct 30 19:45:35.600307 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:35.600145    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq" event={"ID":"d71d8254-621d-4cac-94ea-28c912d5a45b","Type":"ContainerDied","Data":"20239eca8c288234216eab29e978f7e986ebef29c49cde68ed39bab6e49ef441"}
Oct 30 19:45:35.959813 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:35.957919    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67953efd-956bfdb9-4ftfp"
Oct 30 19:45:35.968965 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:35.968941    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq"
Oct 30 19:45:36.150536 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:36.150497    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d71d8254-621d-4cac-94ea-28c912d5a45b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d71d8254-621d-4cac-94ea-28c912d5a45b-openshift-service-ca\") pod \"d71d8254-621d-4cac-94ea-28c912d5a45b\" (UID: \"d71d8254-621d-4cac-94ea-28c912d5a45b\") "
Oct 30 19:45:36.150944 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:36.150916    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d71d8254-621d-4cac-94ea-28c912d5a45b-kube-api-access-55kjq\" (UniqueName: \"kubernetes.io/projected/d71d8254-621d-4cac-94ea-28c912d5a45b-kube-api-access-55kjq\") pod \"d71d8254-621d-4cac-94ea-28c912d5a45b\" (UID: \"d71d8254-621d-4cac-94ea-28c912d5a45b\") "
Oct 30 19:45:36.151231 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:36.151212    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d71d8254-621d-4cac-94ea-28c912d5a45b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d71d8254-621d-4cac-94ea-28c912d5a45b-maas-api-tls\") pod \"d71d8254-621d-4cac-94ea-28c912d5a45b\" (UID: \"d71d8254-621d-4cac-94ea-28c912d5a45b\") "
Oct 30 19:45:36.171966 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:36.152215    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d71d8254-621d-4cac-94ea-28c912d5a45b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d71d8254-621d-4cac-94ea-28c912d5a45b-metrics-certs\") pod \"d71d8254-621d-4cac-94ea-28c912d5a45b\" (UID: \"d71d8254-621d-4cac-94ea-28c912d5a45b\") "
Oct 30 19:45:36.187223 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:36.187136    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d71d8254-621d-4cac-94ea-28c912d5a45b-openshift-service-ca" pod "d71d8254-621d-4cac-94ea-28c912d5a45b" (UID: "d71d8254-621d-4cac-94ea-28c912d5a45b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:36.205228 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:36.205194    5907 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d71d8254-621d-4cac-94ea-28c912d5a45b-openshift-service-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:36.246935 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:36.227943    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d71d8254-621d-4cac-94ea-28c912d5a45b-maas-api-tls" pod "d71d8254-621d-4cac-94ea-28c912d5a45b" (UID: "d71d8254-621d-4cac-94ea-28c912d5a45b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:45:36.246935 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:36.228085    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d71d8254-621d-4cac-94ea-28c912d5a45b-metrics-certs" pod "d71d8254-621d-4cac-94ea-28c912d5a45b" (UID: "d71d8254-621d-4cac-94ea-28c912d5a45b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:45:36.289906 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:36.287084    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d71d8254-621d-4cac-94ea-28c912d5a45b-kube-api-access-55kjq" pod "d71d8254-621d-4cac-94ea-28c912d5a45b" (UID: "d71d8254-621d-4cac-94ea-28c912d5a45b"). InnerVolumeSpecName "kube-api-access-55kjq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:36.316900 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:36.315078    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55kjq\" (UniqueName: \"kubernetes.io/projected/d71d8254-621d-4cac-94ea-28c912d5a45b-kube-api-access-55kjq\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:36.316900 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:36.315110    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d71d8254-621d-4cac-94ea-28c912d5a45b-maas-api-tls\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:36.316900 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:36.315127    5907 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d71d8254-621d-4cac-94ea-28c912d5a45b-metrics-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:36.564922 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:36.564888    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-0c305f-596bs"]
Oct 30 19:45:36.570067 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:36.570039    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w0-0c305f-596bs"]
Oct 30 19:45:36.609820 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:36.609791    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq"
Oct 30 19:45:36.609820 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:36.609806    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq" event={"ID":"d71d8254-621d-4cac-94ea-28c912d5a45b","Type":"ContainerDied","Data":"5b04a77650c9b23832ce02f4d0c68d5eb765cd63c05b43f2f2e81528617c9552"}
Oct 30 19:45:36.610116 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:36.609924    5907 scope.go:122] "RemoveContainer" containerID="20239eca8c288234216eab29e978f7e986ebef29c49cde68ed39bab6e49ef441"
Oct 30 19:45:36.675961 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:36.675932    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq"]
Oct 30 19:45:36.679778 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:36.679755    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-0c305f-6677969c6f-7vhcq"]
Oct 30 19:45:38.293169 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:38.293081    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d71d8254-621d-4cac-94ea-28c912d5a45b" path="/var/lib/kubelet/pods/d71d8254-621d-4cac-94ea-28c912d5a45b/volumes"
Oct 30 19:45:38.293670 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:38.293653    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f7f11c63-7327-488a-8749-d7f6c395de05" path="/var/lib/kubelet/pods/f7f11c63-7327-488a-8749-d7f6c395de05/volumes"
Oct 30 19:45:38.629660 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:38.629623    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-3735b2-6f754b9b8b-b9rgh"]
Oct 30 19:45:38.666060 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:38.666024    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-3735b2-795c89ffc9-rtdm4"]
Oct 30 19:45:38.789329 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:38.789288    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0c305f-5f5c4698b6-kzfnv" podUID="400f0778-d2b1-4ca0-93ec-59672a71e661" containerName="payload-processing" containerID="cri-o://395a779cd3a65c10bce5023afe914c6b30c87943c446e2536321552706001a20" gracePeriod=25
Oct 30 19:45:38.855266 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:38.855214    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0c305f-8677844c-mdcqk" podUID="998e3875-ef86-4381-9d65-87ca3bba0b1d" containerName="payload-pre-processing" containerID="cri-o://5db5e188aa0608047f79609c9bad7a6af1ebabfa5d7589f2444f5089323450c5" gracePeriod=25
Oct 30 19:45:39.263889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:39.261035    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0c305f-8677844c-mdcqk"
Oct 30 19:45:39.396885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:39.375134    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0c305f-5f5c4698b6-kzfnv"
Oct 30 19:45:39.412893 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:39.399671    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/998e3875-ef86-4381-9d65-87ca3bba0b1d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/998e3875-ef86-4381-9d65-87ca3bba0b1d-plugins-config-volume\") pod \"998e3875-ef86-4381-9d65-87ca3bba0b1d\" (UID: \"998e3875-ef86-4381-9d65-87ca3bba0b1d\") "
Oct 30 19:45:39.413314 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:39.413283    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/998e3875-ef86-4381-9d65-87ca3bba0b1d-kube-api-access-ldtk2\" (UniqueName: \"kubernetes.io/projected/998e3875-ef86-4381-9d65-87ca3bba0b1d-kube-api-access-ldtk2\") pod \"998e3875-ef86-4381-9d65-87ca3bba0b1d\" (UID: \"998e3875-ef86-4381-9d65-87ca3bba0b1d\") "
Oct 30 19:45:39.415624 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:39.407379    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/998e3875-ef86-4381-9d65-87ca3bba0b1d-plugins-config-volume" pod "998e3875-ef86-4381-9d65-87ca3bba0b1d" (UID: "998e3875-ef86-4381-9d65-87ca3bba0b1d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:39.419506 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:39.419456    5907 patch_prober.go:29] interesting pod/e2e-api-b-9de441-openshift-default-655b8b8797-ps599 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.205:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:45:39.419732 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:39.419700    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599" podUID="0da36910-7634-4556-b95b-451340c2a584" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.205:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:39.425150 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:39.425109    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/998e3875-ef86-4381-9d65-87ca3bba0b1d-kube-api-access-ldtk2" pod "998e3875-ef86-4381-9d65-87ca3bba0b1d" (UID: "998e3875-ef86-4381-9d65-87ca3bba0b1d"). InnerVolumeSpecName "kube-api-access-ldtk2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:39.528922 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:39.527627    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/400f0778-d2b1-4ca0-93ec-59672a71e661-kube-api-access-z8pdz\" (UniqueName: \"kubernetes.io/projected/400f0778-d2b1-4ca0-93ec-59672a71e661-kube-api-access-z8pdz\") pod \"400f0778-d2b1-4ca0-93ec-59672a71e661\" (UID: \"400f0778-d2b1-4ca0-93ec-59672a71e661\") "
Oct 30 19:45:39.528922 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:39.527820    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/400f0778-d2b1-4ca0-93ec-59672a71e661-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/400f0778-d2b1-4ca0-93ec-59672a71e661-plugins-config-volume\") pod \"400f0778-d2b1-4ca0-93ec-59672a71e661\" (UID: \"400f0778-d2b1-4ca0-93ec-59672a71e661\") "
Oct 30 19:45:39.528922 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:39.528629    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldtk2\" (UniqueName: \"kubernetes.io/projected/998e3875-ef86-4381-9d65-87ca3bba0b1d-kube-api-access-ldtk2\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:39.528922 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:39.528652    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/998e3875-ef86-4381-9d65-87ca3bba0b1d-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:39.529267 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:39.529124    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/400f0778-d2b1-4ca0-93ec-59672a71e661-plugins-config-volume" pod "400f0778-d2b1-4ca0-93ec-59672a71e661" (UID: "400f0778-d2b1-4ca0-93ec-59672a71e661"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:39.533310 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:39.533282    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/400f0778-d2b1-4ca0-93ec-59672a71e661-kube-api-access-z8pdz" pod "400f0778-d2b1-4ca0-93ec-59672a71e661" (UID: "400f0778-d2b1-4ca0-93ec-59672a71e661"). InnerVolumeSpecName "kube-api-access-z8pdz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:39.632067 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:39.632031    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/400f0778-d2b1-4ca0-93ec-59672a71e661-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:39.632067 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:39.632068    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8pdz\" (UniqueName: \"kubernetes.io/projected/400f0778-d2b1-4ca0-93ec-59672a71e661-kube-api-access-z8pdz\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:39.651890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:39.651843    5907 generic.go:356] "Generic (PLEG): container finished" podID="400f0778-d2b1-4ca0-93ec-59672a71e661" containerID="395a779cd3a65c10bce5023afe914c6b30c87943c446e2536321552706001a20" exitCode=0
Oct 30 19:45:39.652071 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:39.651916    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0c305f-5f5c4698b6-kzfnv"
Oct 30 19:45:39.652071 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:39.651948    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0c305f-5f5c4698b6-kzfnv" event={"ID":"400f0778-d2b1-4ca0-93ec-59672a71e661","Type":"ContainerDied","Data":"395a779cd3a65c10bce5023afe914c6b30c87943c446e2536321552706001a20"}
Oct 30 19:45:39.652071 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:39.652044    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0c305f-5f5c4698b6-kzfnv" event={"ID":"400f0778-d2b1-4ca0-93ec-59672a71e661","Type":"ContainerDied","Data":"6c610d640e8c2dec303edc6fb0535182038ebd3ebafa97753090aeb148a18017"}
Oct 30 19:45:39.652242 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:39.652075    5907 scope.go:122] "RemoveContainer" containerID="395a779cd3a65c10bce5023afe914c6b30c87943c446e2536321552706001a20"
Oct 30 19:45:39.654157 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:39.654134    5907 generic.go:356] "Generic (PLEG): container finished" podID="998e3875-ef86-4381-9d65-87ca3bba0b1d" containerID="5db5e188aa0608047f79609c9bad7a6af1ebabfa5d7589f2444f5089323450c5" exitCode=0
Oct 30 19:45:39.654313 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:39.654179    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0c305f-8677844c-mdcqk" event={"ID":"998e3875-ef86-4381-9d65-87ca3bba0b1d","Type":"ContainerDied","Data":"5db5e188aa0608047f79609c9bad7a6af1ebabfa5d7589f2444f5089323450c5"}
Oct 30 19:45:39.654313 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:39.654196    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0c305f-8677844c-mdcqk"
Oct 30 19:45:39.654313 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:39.654217    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0c305f-8677844c-mdcqk" event={"ID":"998e3875-ef86-4381-9d65-87ca3bba0b1d","Type":"ContainerDied","Data":"dd0f4526b83f481459f47a4fba5ef4d8fa3c393d7ed26f3edf9828c379055888"}
Oct 30 19:45:39.666512 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:39.666490    5907 scope.go:122] "RemoveContainer" containerID="395a779cd3a65c10bce5023afe914c6b30c87943c446e2536321552706001a20"
Oct 30 19:45:39.666810 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:45:39.666788    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"395a779cd3a65c10bce5023afe914c6b30c87943c446e2536321552706001a20\": container with ID starting with 395a779cd3a65c10bce5023afe914c6b30c87943c446e2536321552706001a20 not found: ID does not exist" containerID="395a779cd3a65c10bce5023afe914c6b30c87943c446e2536321552706001a20"
Oct 30 19:45:39.666900 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:39.666821    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"395a779cd3a65c10bce5023afe914c6b30c87943c446e2536321552706001a20"} err="failed to get container status \"395a779cd3a65c10bce5023afe914c6b30c87943c446e2536321552706001a20\": rpc error: code = NotFound desc = could not find container \"395a779cd3a65c10bce5023afe914c6b30c87943c446e2536321552706001a20\": container with ID starting with 395a779cd3a65c10bce5023afe914c6b30c87943c446e2536321552706001a20 not found: ID does not exist"
Oct 30 19:45:39.666900 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:39.666846    5907 scope.go:122] "RemoveContainer" containerID="5db5e188aa0608047f79609c9bad7a6af1ebabfa5d7589f2444f5089323450c5"
Oct 30 19:45:39.677240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:39.677215    5907 scope.go:122] "RemoveContainer" containerID="5db5e188aa0608047f79609c9bad7a6af1ebabfa5d7589f2444f5089323450c5"
Oct 30 19:45:39.677541 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:45:39.677518    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5db5e188aa0608047f79609c9bad7a6af1ebabfa5d7589f2444f5089323450c5\": container with ID starting with 5db5e188aa0608047f79609c9bad7a6af1ebabfa5d7589f2444f5089323450c5 not found: ID does not exist" containerID="5db5e188aa0608047f79609c9bad7a6af1ebabfa5d7589f2444f5089323450c5"
Oct 30 19:45:39.677602 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:39.677549    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5db5e188aa0608047f79609c9bad7a6af1ebabfa5d7589f2444f5089323450c5"} err="failed to get container status \"5db5e188aa0608047f79609c9bad7a6af1ebabfa5d7589f2444f5089323450c5\": rpc error: code = NotFound desc = could not find container \"5db5e188aa0608047f79609c9bad7a6af1ebabfa5d7589f2444f5089323450c5\": container with ID starting with 5db5e188aa0608047f79609c9bad7a6af1ebabfa5d7589f2444f5089323450c5 not found: ID does not exist"
Oct 30 19:45:39.713077 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:39.713043    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-0c305f-5f5c4698b6-kzfnv"]
Oct 30 19:45:39.716704 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:39.716681    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-0c305f-5f5c4698b6-kzfnv"]
Oct 30 19:45:39.758703 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:39.758677    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0c305f-8677844c-mdcqk"]
Oct 30 19:45:39.762451 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:39.762424    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0c305f-8677844c-mdcqk"]
Oct 30 19:45:40.290293 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:40.290258    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="400f0778-d2b1-4ca0-93ec-59672a71e661" path="/var/lib/kubelet/pods/400f0778-d2b1-4ca0-93ec-59672a71e661/volumes"
Oct 30 19:45:40.290900 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:40.290873    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="998e3875-ef86-4381-9d65-87ca3bba0b1d" path="/var/lib/kubelet/pods/998e3875-ef86-4381-9d65-87ca3bba0b1d/volumes"
Oct 30 19:45:41.674500 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:41.674466    5907 generic.go:356] "Generic (PLEG): container finished" podID="0da36910-7634-4556-b95b-451340c2a584" containerID="32588ff3a44c0194908475273296014ab5cba4c031e6f156a04a7bda33d45605" exitCode=0
Oct 30 19:45:41.674994 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:41.674541    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599" event={"ID":"0da36910-7634-4556-b95b-451340c2a584","Type":"ContainerDied","Data":"32588ff3a44c0194908475273296014ab5cba4c031e6f156a04a7bda33d45605"}
Oct 30 19:45:41.675043 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:41.675006    5907 scope.go:122] "RemoveContainer" containerID="32588ff3a44c0194908475273296014ab5cba4c031e6f156a04a7bda33d45605"
Oct 30 19:45:41.676615 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:41.676580    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:45:41.676695 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:41.676657    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:45:41.676733 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:41.676699    5907 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 19:45:42.684682 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:42.684649    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599" event={"ID":"0da36910-7634-4556-b95b-451340c2a584","Type":"ContainerStarted","Data":"6b7a2f4c29214fc5835150fb7c7903225270f1c9bedfaf731cebcc6551b920c6"}
Oct 30 19:45:42.685714 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:42.685690    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:45:42.685938 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:42.685915    5907 patch_prober.go:29] interesting pod/e2e-api-b-9de441-openshift-default-655b8b8797-ps599 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.205:15021/healthz/ready\": dial tcp 10.134.0.205:15021: connect: connection refused" start-of-body=
Oct 30 19:45:42.686032 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:42.685970    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599" podUID="0da36910-7634-4556-b95b-451340c2a584" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.205:15021/healthz/ready\": dial tcp 10.134.0.205:15021: connect: connection refused"
Oct 30 19:45:43.631891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:43.631623    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w2-3735b2-6f754b9b8b-b9rgh" podUID="b1714315-7df0-4860-8759-bdfab7c13824" containerName="payload-processing" containerID="cri-o://4997148dc75040d79541acda6e57517025cdce6a421afddc7216ba87cb6f8b43" gracePeriod=25
Oct 30 19:45:43.667009 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:43.666973    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-3735b2-795c89ffc9-rtdm4" podUID="97e31b29-cb25-492c-bd66-fcab4b422ab9" containerName="payload-pre-processing" containerID="cri-o://69a5752e0cc2ce257d207fec4bb9b500ae53910687a54933a950146d925642e3" gracePeriod=25
Oct 30 19:45:43.686444 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:43.686413    5907 patch_prober.go:29] interesting pod/e2e-api-b-9de441-openshift-default-655b8b8797-ps599 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.205:15021/healthz/ready\": dial tcp 10.134.0.205:15021: connect: connection refused" start-of-body=
Oct 30 19:45:43.687087 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:43.686467    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599" podUID="0da36910-7634-4556-b95b-451340c2a584" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.205:15021/healthz/ready\": dial tcp 10.134.0.205:15021: connect: connection refused"
Oct 30 19:45:43.761903 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:43.759219    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k"]
Oct 30 19:45:43.761903 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:43.759583    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k" podUID="3f957a7e-8e0a-477b-a4da-0d098f7ab5c2" containerName="maas-api" containerID="cri-o://cf0c9fe3897c7ecbc6ee1acc471b246d6a45fb9d16f184d87359d897f4d9144d" gracePeriod=30
Oct 30 19:45:44.495182 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.495094    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-381d04c5-s8l9s"]
Oct 30 19:45:44.496359 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.496331    5907 cpu_manager.go:414] "containerMap: removing container" podUID="998e3875-ef86-4381-9d65-87ca3bba0b1d" containerName="payload-pre-processing"
Oct 30 19:45:44.496359 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.496358    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="998e3875-ef86-4381-9d65-87ca3bba0b1d" containerName="payload-pre-processing"
Oct 30 19:45:44.496565 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.496412    5907 cpu_manager.go:414] "containerMap: removing container" podUID="400f0778-d2b1-4ca0-93ec-59672a71e661" containerName="payload-processing"
Oct 30 19:45:44.496565 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.496421    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="400f0778-d2b1-4ca0-93ec-59672a71e661" containerName="payload-processing"
Oct 30 19:45:44.496565 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.496457    5907 cpu_manager.go:414] "containerMap: removing container" podUID="d71d8254-621d-4cac-94ea-28c912d5a45b" containerName="maas-api"
Oct 30 19:45:44.496565 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.496466    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d71d8254-621d-4cac-94ea-28c912d5a45b" containerName="maas-api"
Oct 30 19:45:44.496788 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.496680    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="998e3875-ef86-4381-9d65-87ca3bba0b1d" containerName="payload-pre-processing"
Oct 30 19:45:44.496788 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.496705    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="400f0778-d2b1-4ca0-93ec-59672a71e661" containerName="payload-processing"
Oct 30 19:45:44.496788 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.496720    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="d71d8254-621d-4cac-94ea-28c912d5a45b" containerName="maas-api"
Oct 30 19:45:44.506002 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.505961    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae7b11b0-xkbwp"]
Oct 30 19:45:44.506343 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.506312    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-381d04c5-s8l9s"
Oct 30 19:45:44.530891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.527811    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:45:44.530891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.528519    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:45:44.539885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.537030    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-381d04c5-s8l9s"]
Oct 30 19:45:44.539885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.537082    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae7b11b0-xkbwp"]
Oct 30 19:45:44.539885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.537180    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae7b11b0-xkbwp"
Oct 30 19:45:44.598244 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.598219    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k"
Oct 30 19:45:44.659147 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.659109    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3b1170ae-7298-4455-8d90-8e08d345b5a6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ae7b11b0-xkbwp\" (UID: \"3b1170ae-7298-4455-8d90-8e08d345b5a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae7b11b0-xkbwp"
Oct 30 19:45:44.659358 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.659251    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3b1170ae-7298-4455-8d90-8e08d345b5a6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ae7b11b0-xkbwp\" (UID: \"3b1170ae-7298-4455-8d90-8e08d345b5a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae7b11b0-xkbwp"
Oct 30 19:45:44.659358 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.659296    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwcms\" (UniqueName: \"kubernetes.io/projected/3b1170ae-7298-4455-8d90-8e08d345b5a6-kube-api-access-gwcms\") pod \"maas-api-revoke-sub-ae7b11b0-xkbwp\" (UID: \"3b1170ae-7298-4455-8d90-8e08d345b5a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae7b11b0-xkbwp"
Oct 30 19:45:44.659539 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.659514    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlr6h\" (UniqueName: \"kubernetes.io/projected/d4e03ee0-1d1a-492a-b272-4b131d39dc4b-kube-api-access-jlr6h\") pod \"maas-api-revoke-sub-381d04c5-s8l9s\" (UID: \"d4e03ee0-1d1a-492a-b272-4b131d39dc4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-381d04c5-s8l9s"
Oct 30 19:45:44.659670 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.659571    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d4e03ee0-1d1a-492a-b272-4b131d39dc4b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-381d04c5-s8l9s\" (UID: \"d4e03ee0-1d1a-492a-b272-4b131d39dc4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-381d04c5-s8l9s"
Oct 30 19:45:44.659736 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.659701    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d4e03ee0-1d1a-492a-b272-4b131d39dc4b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-381d04c5-s8l9s\" (UID: \"d4e03ee0-1d1a-492a-b272-4b131d39dc4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-381d04c5-s8l9s"
Oct 30 19:45:44.687162 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.687132    5907 patch_prober.go:29] interesting pod/e2e-api-b-9de441-openshift-default-655b8b8797-ps599 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.205:15021/healthz/ready\": dial tcp 10.134.0.205:15021: connect: connection refused" start-of-body=
Oct 30 19:45:44.687521 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.687178    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599" podUID="0da36910-7634-4556-b95b-451340c2a584" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.205:15021/healthz/ready\": dial tcp 10.134.0.205:15021: connect: connection refused"
Oct 30 19:45:44.737057 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.736590    5907 generic.go:356] "Generic (PLEG): container finished" podID="3f957a7e-8e0a-477b-a4da-0d098f7ab5c2" containerID="cf0c9fe3897c7ecbc6ee1acc471b246d6a45fb9d16f184d87359d897f4d9144d" exitCode=0
Oct 30 19:45:44.737057 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.736668    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k"
Oct 30 19:45:44.737057 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.736923    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k" event={"ID":"3f957a7e-8e0a-477b-a4da-0d098f7ab5c2","Type":"ContainerDied","Data":"cf0c9fe3897c7ecbc6ee1acc471b246d6a45fb9d16f184d87359d897f4d9144d"}
Oct 30 19:45:44.737057 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.736980    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k" event={"ID":"3f957a7e-8e0a-477b-a4da-0d098f7ab5c2","Type":"ContainerDied","Data":"f6c49f869194767c28f49e90089ea42f61b3687a8308275ad7b7073bb2d34cb2"}
Oct 30 19:45:44.737057 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.737004    5907 scope.go:122] "RemoveContainer" containerID="cf0c9fe3897c7ecbc6ee1acc471b246d6a45fb9d16f184d87359d897f4d9144d"
Oct 30 19:45:44.749191 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.749163    5907 scope.go:122] "RemoveContainer" containerID="cf0c9fe3897c7ecbc6ee1acc471b246d6a45fb9d16f184d87359d897f4d9144d"
Oct 30 19:45:44.749690 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:45:44.749663    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf0c9fe3897c7ecbc6ee1acc471b246d6a45fb9d16f184d87359d897f4d9144d\": container with ID starting with cf0c9fe3897c7ecbc6ee1acc471b246d6a45fb9d16f184d87359d897f4d9144d not found: ID does not exist" containerID="cf0c9fe3897c7ecbc6ee1acc471b246d6a45fb9d16f184d87359d897f4d9144d"
Oct 30 19:45:44.749797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.749699    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf0c9fe3897c7ecbc6ee1acc471b246d6a45fb9d16f184d87359d897f4d9144d"} err="failed to get container status \"cf0c9fe3897c7ecbc6ee1acc471b246d6a45fb9d16f184d87359d897f4d9144d\": rpc error: code = NotFound desc = could not find container \"cf0c9fe3897c7ecbc6ee1acc471b246d6a45fb9d16f184d87359d897f4d9144d\": container with ID starting with cf0c9fe3897c7ecbc6ee1acc471b246d6a45fb9d16f184d87359d897f4d9144d not found: ID does not exist"
Oct 30 19:45:44.761010 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.760982    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3f957a7e-8e0a-477b-a4da-0d098f7ab5c2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f957a7e-8e0a-477b-a4da-0d098f7ab5c2-metrics-certs\") pod \"3f957a7e-8e0a-477b-a4da-0d098f7ab5c2\" (UID: \"3f957a7e-8e0a-477b-a4da-0d098f7ab5c2\") "
Oct 30 19:45:44.761010 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.761011    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3f957a7e-8e0a-477b-a4da-0d098f7ab5c2-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3f957a7e-8e0a-477b-a4da-0d098f7ab5c2-maas-api-tls\") pod \"3f957a7e-8e0a-477b-a4da-0d098f7ab5c2\" (UID: \"3f957a7e-8e0a-477b-a4da-0d098f7ab5c2\") "
Oct 30 19:45:44.761259 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.761247    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3f957a7e-8e0a-477b-a4da-0d098f7ab5c2-kube-api-access-z6h7s\" (UniqueName: \"kubernetes.io/projected/3f957a7e-8e0a-477b-a4da-0d098f7ab5c2-kube-api-access-z6h7s\") pod \"3f957a7e-8e0a-477b-a4da-0d098f7ab5c2\" (UID: \"3f957a7e-8e0a-477b-a4da-0d098f7ab5c2\") "
Oct 30 19:45:44.761339 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.761296    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3f957a7e-8e0a-477b-a4da-0d098f7ab5c2-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f957a7e-8e0a-477b-a4da-0d098f7ab5c2-openshift-service-ca\") pod \"3f957a7e-8e0a-477b-a4da-0d098f7ab5c2\" (UID: \"3f957a7e-8e0a-477b-a4da-0d098f7ab5c2\") "
Oct 30 19:45:44.761578 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.761557    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlr6h\" (UniqueName: \"kubernetes.io/projected/d4e03ee0-1d1a-492a-b272-4b131d39dc4b-kube-api-access-jlr6h\") pod \"maas-api-revoke-sub-381d04c5-s8l9s\" (UID: \"d4e03ee0-1d1a-492a-b272-4b131d39dc4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-381d04c5-s8l9s"
Oct 30 19:45:44.761697 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.761607    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d4e03ee0-1d1a-492a-b272-4b131d39dc4b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-381d04c5-s8l9s\" (UID: \"d4e03ee0-1d1a-492a-b272-4b131d39dc4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-381d04c5-s8l9s"
Oct 30 19:45:44.761767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.761695    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d4e03ee0-1d1a-492a-b272-4b131d39dc4b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-381d04c5-s8l9s\" (UID: \"d4e03ee0-1d1a-492a-b272-4b131d39dc4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-381d04c5-s8l9s"
Oct 30 19:45:44.761767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.761722    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f957a7e-8e0a-477b-a4da-0d098f7ab5c2-openshift-service-ca" pod "3f957a7e-8e0a-477b-a4da-0d098f7ab5c2" (UID: "3f957a7e-8e0a-477b-a4da-0d098f7ab5c2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:44.762156 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.762135    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3b1170ae-7298-4455-8d90-8e08d345b5a6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ae7b11b0-xkbwp\" (UID: \"3b1170ae-7298-4455-8d90-8e08d345b5a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae7b11b0-xkbwp"
Oct 30 19:45:44.762274 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.762245    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3b1170ae-7298-4455-8d90-8e08d345b5a6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ae7b11b0-xkbwp\" (UID: \"3b1170ae-7298-4455-8d90-8e08d345b5a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae7b11b0-xkbwp"
Oct 30 19:45:44.762348 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.762289    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwcms\" (UniqueName: \"kubernetes.io/projected/3b1170ae-7298-4455-8d90-8e08d345b5a6-kube-api-access-gwcms\") pod \"maas-api-revoke-sub-ae7b11b0-xkbwp\" (UID: \"3b1170ae-7298-4455-8d90-8e08d345b5a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae7b11b0-xkbwp"
Oct 30 19:45:44.762424 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.762412    5907 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f957a7e-8e0a-477b-a4da-0d098f7ab5c2-openshift-service-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:44.762577 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.762556    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d4e03ee0-1d1a-492a-b272-4b131d39dc4b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-381d04c5-s8l9s\" (UID: \"d4e03ee0-1d1a-492a-b272-4b131d39dc4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-381d04c5-s8l9s"
Oct 30 19:45:44.763097 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.763075    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3b1170ae-7298-4455-8d90-8e08d345b5a6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ae7b11b0-xkbwp\" (UID: \"3b1170ae-7298-4455-8d90-8e08d345b5a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae7b11b0-xkbwp"
Oct 30 19:45:44.764021 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.763993    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f957a7e-8e0a-477b-a4da-0d098f7ab5c2-metrics-certs" pod "3f957a7e-8e0a-477b-a4da-0d098f7ab5c2" (UID: "3f957a7e-8e0a-477b-a4da-0d098f7ab5c2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:45:44.764137 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.763994    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f957a7e-8e0a-477b-a4da-0d098f7ab5c2-kube-api-access-z6h7s" pod "3f957a7e-8e0a-477b-a4da-0d098f7ab5c2" (UID: "3f957a7e-8e0a-477b-a4da-0d098f7ab5c2"). InnerVolumeSpecName "kube-api-access-z6h7s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:44.764294 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.764265    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d4e03ee0-1d1a-492a-b272-4b131d39dc4b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-381d04c5-s8l9s\" (UID: \"d4e03ee0-1d1a-492a-b272-4b131d39dc4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-381d04c5-s8l9s"
Oct 30 19:45:44.764536 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.764514    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f957a7e-8e0a-477b-a4da-0d098f7ab5c2-maas-api-tls" pod "3f957a7e-8e0a-477b-a4da-0d098f7ab5c2" (UID: "3f957a7e-8e0a-477b-a4da-0d098f7ab5c2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:45:44.764956 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.764940    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3b1170ae-7298-4455-8d90-8e08d345b5a6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ae7b11b0-xkbwp\" (UID: \"3b1170ae-7298-4455-8d90-8e08d345b5a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae7b11b0-xkbwp"
Oct 30 19:45:44.773500 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.773430    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-086f2a-flxnc"]
Oct 30 19:45:44.779634 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.779612    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-086f2a-flxnc"]
Oct 30 19:45:44.795659 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.792761    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwcms\" (UniqueName: \"kubernetes.io/projected/3b1170ae-7298-4455-8d90-8e08d345b5a6-kube-api-access-gwcms\") pod \"maas-api-revoke-sub-ae7b11b0-xkbwp\" (UID: \"3b1170ae-7298-4455-8d90-8e08d345b5a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae7b11b0-xkbwp"
Oct 30 19:45:44.795659 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.794344    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlr6h\" (UniqueName: \"kubernetes.io/projected/d4e03ee0-1d1a-492a-b272-4b131d39dc4b-kube-api-access-jlr6h\") pod \"maas-api-revoke-sub-381d04c5-s8l9s\" (UID: \"d4e03ee0-1d1a-492a-b272-4b131d39dc4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-381d04c5-s8l9s"
Oct 30 19:45:44.850888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.840378    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-381d04c5-s8l9s"
Oct 30 19:45:44.890964 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.878572    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6h7s\" (UniqueName: \"kubernetes.io/projected/3f957a7e-8e0a-477b-a4da-0d098f7ab5c2-kube-api-access-z6h7s\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:44.910238 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.901050    5907 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f957a7e-8e0a-477b-a4da-0d098f7ab5c2-metrics-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:44.910238 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.901079    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3f957a7e-8e0a-477b-a4da-0d098f7ab5c2-maas-api-tls\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:44.910238 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:44.891993    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae7b11b0-xkbwp"
Oct 30 19:45:45.168692 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:45.168636    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k"]
Oct 30 19:45:45.174232 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:45.174173    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-086f2a-7d7df4489f-bnc9k"]
Oct 30 19:45:45.431886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:45.410824    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-381d04c5-s8l9s"]
Oct 30 19:45:45.436020 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:45:45.435976    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4e03ee0_1d1a_492a_b272_4b131d39dc4b.slice/crio-e23c9f0d464be2ed894678c999450afe34c4151f9ec35ead7b1dedac0f8851cd WatchSource:0}: Error finding container e23c9f0d464be2ed894678c999450afe34c4151f9ec35ead7b1dedac0f8851cd: Status 404 returned error can't find the container with id e23c9f0d464be2ed894678c999450afe34c4151f9ec35ead7b1dedac0f8851cd
Oct 30 19:45:45.441428 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:45.441404    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae7b11b0-xkbwp"]
Oct 30 19:45:45.443533 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:45:45.443495    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b1170ae_7298_4455_8d90_8e08d345b5a6.slice/crio-1bf9dbee4281826b17e19535ad7f1232c3cd4d9878c787fa8fc8d2b773b86312 WatchSource:0}: Error finding container 1bf9dbee4281826b17e19535ad7f1232c3cd4d9878c787fa8fc8d2b773b86312: Status 404 returned error can't find the container with id 1bf9dbee4281826b17e19535ad7f1232c3cd4d9878c787fa8fc8d2b773b86312
Oct 30 19:45:45.687018 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:45.686930    5907 patch_prober.go:29] interesting pod/e2e-api-b-9de441-openshift-default-655b8b8797-ps599 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.205:15021/healthz/ready\": dial tcp 10.134.0.205:15021: connect: connection refused" start-of-body=
Oct 30 19:45:45.687018 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:45.686978    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599" podUID="0da36910-7634-4556-b95b-451340c2a584" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.205:15021/healthz/ready\": dial tcp 10.134.0.205:15021: connect: connection refused"
Oct 30 19:45:45.753852 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:45.753809    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae7b11b0-xkbwp" event={"ID":"3b1170ae-7298-4455-8d90-8e08d345b5a6","Type":"ContainerStarted","Data":"1bf9dbee4281826b17e19535ad7f1232c3cd4d9878c787fa8fc8d2b773b86312"}
Oct 30 19:45:45.757945 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:45.757912    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-381d04c5-s8l9s" event={"ID":"d4e03ee0-1d1a-492a-b272-4b131d39dc4b","Type":"ContainerStarted","Data":"e23c9f0d464be2ed894678c999450afe34c4151f9ec35ead7b1dedac0f8851cd"}
Oct 30 19:45:46.290641 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:46.290560    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3f957a7e-8e0a-477b-a4da-0d098f7ab5c2" path="/var/lib/kubelet/pods/3f957a7e-8e0a-477b-a4da-0d098f7ab5c2/volumes"
Oct 30 19:45:46.291123 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:46.291106    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d6dc0922-1322-4dfe-b13e-b71af1f9d966" path="/var/lib/kubelet/pods/d6dc0922-1322-4dfe-b13e-b71af1f9d966/volumes"
Oct 30 19:45:46.689351 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:46.685834    5907 patch_prober.go:29] interesting pod/e2e-api-b-9de441-openshift-default-655b8b8797-ps599 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.205:15021/healthz/ready\": dial tcp 10.134.0.205:15021: connect: connection refused" start-of-body=
Oct 30 19:45:46.689351 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:46.685924    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599" podUID="0da36910-7634-4556-b95b-451340c2a584" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.205:15021/healthz/ready\": dial tcp 10.134.0.205:15021: connect: connection refused"
Oct 30 19:45:46.768456 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:46.768413    5907 generic.go:356] "Generic (PLEG): container finished" podID="d4e03ee0-1d1a-492a-b272-4b131d39dc4b" containerID="f2b5d3faef45ac2468ab6e4771cc113d6f2c70a0c9f6a6dd85a1dbab4c4bacdb" exitCode=0
Oct 30 19:45:46.768792 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:46.768671    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-381d04c5-s8l9s" event={"ID":"d4e03ee0-1d1a-492a-b272-4b131d39dc4b","Type":"ContainerDied","Data":"f2b5d3faef45ac2468ab6e4771cc113d6f2c70a0c9f6a6dd85a1dbab4c4bacdb"}
Oct 30 19:45:46.771263 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:46.771230    5907 generic.go:356] "Generic (PLEG): container finished" podID="3b1170ae-7298-4455-8d90-8e08d345b5a6" containerID="741980e41e22869eff3d026aa4c5fec6a774ad5f51aa951ebd0391ba76c5d765" exitCode=0
Oct 30 19:45:46.771382 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:46.771289    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae7b11b0-xkbwp" event={"ID":"3b1170ae-7298-4455-8d90-8e08d345b5a6","Type":"ContainerDied","Data":"741980e41e22869eff3d026aa4c5fec6a774ad5f51aa951ebd0391ba76c5d765"}
Oct 30 19:45:47.506044 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:45:47.506009    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:45:47.686496 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:47.686454    5907 patch_prober.go:29] interesting pod/e2e-api-b-9de441-openshift-default-655b8b8797-ps599 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.205:15021/healthz/ready\": dial tcp 10.134.0.205:15021: connect: connection refused" start-of-body=
Oct 30 19:45:47.686665 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:47.686516    5907 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599" podUID="0da36910-7634-4556-b95b-451340c2a584" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.205:15021/healthz/ready\": dial tcp 10.134.0.205:15021: connect: connection refused"
Oct 30 19:45:48.043500 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.043394    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-3735b2-795c89ffc9-rtdm4" podUID="97e31b29-cb25-492c-bd66-fcab4b422ab9" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:45:48.044960 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.044340    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-3735b2-6f754b9b8b-b9rgh" podUID="b1714315-7df0-4860-8759-bdfab7c13824" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:45:48.049754 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.049732    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae7b11b0-xkbwp"
Oct 30 19:45:48.074164 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.074122    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" containerID="cri-o://740de951966db6ed02d8d9cf86d300a1468e8033e14afab8f08c503ab737e23c" gracePeriod=45
Oct 30 19:45:48.144685 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.144649    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3b1170ae-7298-4455-8d90-8e08d345b5a6-kube-api-access-gwcms\" (UniqueName: \"kubernetes.io/projected/3b1170ae-7298-4455-8d90-8e08d345b5a6-kube-api-access-gwcms\") pod \"3b1170ae-7298-4455-8d90-8e08d345b5a6\" (UID: \"3b1170ae-7298-4455-8d90-8e08d345b5a6\") "
Oct 30 19:45:48.144908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.144738    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3b1170ae-7298-4455-8d90-8e08d345b5a6-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3b1170ae-7298-4455-8d90-8e08d345b5a6-maas-api-xks-ca\") pod \"3b1170ae-7298-4455-8d90-8e08d345b5a6\" (UID: \"3b1170ae-7298-4455-8d90-8e08d345b5a6\") "
Oct 30 19:45:48.144997 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.144946    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3b1170ae-7298-4455-8d90-8e08d345b5a6-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3b1170ae-7298-4455-8d90-8e08d345b5a6-maas-api-openshift-ca\") pod \"3b1170ae-7298-4455-8d90-8e08d345b5a6\" (UID: \"3b1170ae-7298-4455-8d90-8e08d345b5a6\") "
Oct 30 19:45:48.145395 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.145362    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b1170ae-7298-4455-8d90-8e08d345b5a6-maas-api-openshift-ca" pod "3b1170ae-7298-4455-8d90-8e08d345b5a6" (UID: "3b1170ae-7298-4455-8d90-8e08d345b5a6"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:48.145878 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.145834    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3b1170ae-7298-4455-8d90-8e08d345b5a6-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:48.147201 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.147176    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b1170ae-7298-4455-8d90-8e08d345b5a6-maas-api-xks-ca" pod "3b1170ae-7298-4455-8d90-8e08d345b5a6" (UID: "3b1170ae-7298-4455-8d90-8e08d345b5a6"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:45:48.147477 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.147456    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b1170ae-7298-4455-8d90-8e08d345b5a6-kube-api-access-gwcms" pod "3b1170ae-7298-4455-8d90-8e08d345b5a6" (UID: "3b1170ae-7298-4455-8d90-8e08d345b5a6"). InnerVolumeSpecName "kube-api-access-gwcms". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:48.169958 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.169700    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-381d04c5-s8l9s"
Oct 30 19:45:48.248649 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.247145    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d4e03ee0-1d1a-492a-b272-4b131d39dc4b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d4e03ee0-1d1a-492a-b272-4b131d39dc4b-maas-api-openshift-ca\") pod \"d4e03ee0-1d1a-492a-b272-4b131d39dc4b\" (UID: \"d4e03ee0-1d1a-492a-b272-4b131d39dc4b\") "
Oct 30 19:45:48.248649 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.247264    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d4e03ee0-1d1a-492a-b272-4b131d39dc4b-kube-api-access-jlr6h\" (UniqueName: \"kubernetes.io/projected/d4e03ee0-1d1a-492a-b272-4b131d39dc4b-kube-api-access-jlr6h\") pod \"d4e03ee0-1d1a-492a-b272-4b131d39dc4b\" (UID: \"d4e03ee0-1d1a-492a-b272-4b131d39dc4b\") "
Oct 30 19:45:48.248649 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.247412    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d4e03ee0-1d1a-492a-b272-4b131d39dc4b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d4e03ee0-1d1a-492a-b272-4b131d39dc4b-maas-api-xks-ca\") pod \"d4e03ee0-1d1a-492a-b272-4b131d39dc4b\" (UID: \"d4e03ee0-1d1a-492a-b272-4b131d39dc4b\") "
Oct 30 19:45:48.248649 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.248136    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3b1170ae-7298-4455-8d90-8e08d345b5a6-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:48.248649 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.248159    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwcms\" (UniqueName: \"kubernetes.io/projected/3b1170ae-7298-4455-8d90-8e08d345b5a6-kube-api-access-gwcms\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:48.249628 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.249304    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4e03ee0-1d1a-492a-b272-4b131d39dc4b-maas-api-openshift-ca" pod "d4e03ee0-1d1a-492a-b272-4b131d39dc4b" (UID: "d4e03ee0-1d1a-492a-b272-4b131d39dc4b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:48.253664 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.253608    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4e03ee0-1d1a-492a-b272-4b131d39dc4b-maas-api-xks-ca" pod "d4e03ee0-1d1a-492a-b272-4b131d39dc4b" (UID: "d4e03ee0-1d1a-492a-b272-4b131d39dc4b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:45:48.255968 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.255931    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4e03ee0-1d1a-492a-b272-4b131d39dc4b-kube-api-access-jlr6h" pod "d4e03ee0-1d1a-492a-b272-4b131d39dc4b" (UID: "d4e03ee0-1d1a-492a-b272-4b131d39dc4b"). InnerVolumeSpecName "kube-api-access-jlr6h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:48.349910 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.349850    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d4e03ee0-1d1a-492a-b272-4b131d39dc4b-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:48.350165 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.350145    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlr6h\" (UniqueName: \"kubernetes.io/projected/d4e03ee0-1d1a-492a-b272-4b131d39dc4b-kube-api-access-jlr6h\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:48.350286 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.350270    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d4e03ee0-1d1a-492a-b272-4b131d39dc4b-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:48.392918 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.392890    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:45:48.450828 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.450797    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-home\" (UniqueName: \"kubernetes.io/empty-dir/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-home\") pod \"0832aa0a-f1ee-488f-a44e-4b56726e9b7f\" (UID: \"0832aa0a-f1ee-488f-a44e-4b56726e9b7f\") "
Oct 30 19:45:48.451039 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.450898    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-tls-certs\" (UniqueName: \"kubernetes.io/secret/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-tls-certs\") pod \"0832aa0a-f1ee-488f-a44e-4b56726e9b7f\" (UID: \"0832aa0a-f1ee-488f-a44e-4b56726e9b7f\") "
Oct 30 19:45:48.451039 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.450943    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-model-cache\") pod \"0832aa0a-f1ee-488f-a44e-4b56726e9b7f\" (UID: \"0832aa0a-f1ee-488f-a44e-4b56726e9b7f\") "
Oct 30 19:45:48.451039 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.451018    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-dshm\") pod \"0832aa0a-f1ee-488f-a44e-4b56726e9b7f\" (UID: \"0832aa0a-f1ee-488f-a44e-4b56726e9b7f\") "
Oct 30 19:45:48.451231 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.451065    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-kube-api-access-pt47b\" (UniqueName: \"kubernetes.io/projected/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-kube-api-access-pt47b\") pod \"0832aa0a-f1ee-488f-a44e-4b56726e9b7f\" (UID: \"0832aa0a-f1ee-488f-a44e-4b56726e9b7f\") "
Oct 30 19:45:48.451231 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.451064    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-home" pod "0832aa0a-f1ee-488f-a44e-4b56726e9b7f" (UID: "0832aa0a-f1ee-488f-a44e-4b56726e9b7f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:48.451231 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.451114    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-tmp-dir\") pod \"0832aa0a-f1ee-488f-a44e-4b56726e9b7f\" (UID: \"0832aa0a-f1ee-488f-a44e-4b56726e9b7f\") "
Oct 30 19:45:48.451396 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.451285    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-model-cache" pod "0832aa0a-f1ee-488f-a44e-4b56726e9b7f" (UID: "0832aa0a-f1ee-488f-a44e-4b56726e9b7f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:48.451468 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.451402    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-tmp-dir" pod "0832aa0a-f1ee-488f-a44e-4b56726e9b7f" (UID: "0832aa0a-f1ee-488f-a44e-4b56726e9b7f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:48.451826 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.451799    5907 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-tmp-dir\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:48.451826 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.451828    5907 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-home\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:48.452078 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.451843    5907 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-model-cache\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:48.453452 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.453431    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-dshm" pod "0832aa0a-f1ee-488f-a44e-4b56726e9b7f" (UID: "0832aa0a-f1ee-488f-a44e-4b56726e9b7f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:48.453561 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.453489    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-tls-certs" pod "0832aa0a-f1ee-488f-a44e-4b56726e9b7f" (UID: "0832aa0a-f1ee-488f-a44e-4b56726e9b7f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:45:48.453561 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.453552    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-kube-api-access-pt47b" pod "0832aa0a-f1ee-488f-a44e-4b56726e9b7f" (UID: "0832aa0a-f1ee-488f-a44e-4b56726e9b7f"). InnerVolumeSpecName "kube-api-access-pt47b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:48.554014 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.553978    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pt47b\" (UniqueName: \"kubernetes.io/projected/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-kube-api-access-pt47b\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:48.554014 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.554008    5907 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-tls-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:48.554014 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.554018    5907 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0832aa0a-f1ee-488f-a44e-4b56726e9b7f-dshm\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:48.690490 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.690458    5907 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:45:48.690999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.690982    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:45:48.691704 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.691682    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:45:48.739770 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.739739    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3add95-z49q7"]
Oct 30 19:45:48.740549 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.740534    5907 cpu_manager.go:414] "containerMap: removing container" podUID="3f957a7e-8e0a-477b-a4da-0d098f7ab5c2" containerName="maas-api"
Oct 30 19:45:48.740601 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.740551    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3f957a7e-8e0a-477b-a4da-0d098f7ab5c2" containerName="maas-api"
Oct 30 19:45:48.740601 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.740590    5907 cpu_manager.go:414] "containerMap: removing container" podUID="3b1170ae-7298-4455-8d90-8e08d345b5a6" containerName="revoke-keys"
Oct 30 19:45:48.740601 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.740596    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3b1170ae-7298-4455-8d90-8e08d345b5a6" containerName="revoke-keys"
Oct 30 19:45:48.740698 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.740604    5907 cpu_manager.go:414] "containerMap: removing container" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main"
Oct 30 19:45:48.740698 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.740609    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main"
Oct 30 19:45:48.740698 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.740630    5907 cpu_manager.go:414] "containerMap: removing container" podUID="d4e03ee0-1d1a-492a-b272-4b131d39dc4b" containerName="revoke-keys"
Oct 30 19:45:48.740698 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.740634    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d4e03ee0-1d1a-492a-b272-4b131d39dc4b" containerName="revoke-keys"
Oct 30 19:45:48.740823 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.740750    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="d4e03ee0-1d1a-492a-b272-4b131d39dc4b" containerName="revoke-keys"
Oct 30 19:45:48.740823 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.740763    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="3f957a7e-8e0a-477b-a4da-0d098f7ab5c2" containerName="maas-api"
Oct 30 19:45:48.740823 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.740776    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="3b1170ae-7298-4455-8d90-8e08d345b5a6" containerName="revoke-keys"
Oct 30 19:45:48.740823 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.740783    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main"
Oct 30 19:45:48.744830 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.744813    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3add95-z49q7"
Oct 30 19:45:48.780770 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.780736    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3add95-z49q7"]
Oct 30 19:45:48.795399 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.795363    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-381d04c5-s8l9s"
Oct 30 19:45:48.796174 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.796147    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-381d04c5-s8l9s" event={"ID":"d4e03ee0-1d1a-492a-b272-4b131d39dc4b","Type":"ContainerDied","Data":"e23c9f0d464be2ed894678c999450afe34c4151f9ec35ead7b1dedac0f8851cd"}
Oct 30 19:45:48.796287 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.796197    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e23c9f0d464be2ed894678c999450afe34c4151f9ec35ead7b1dedac0f8851cd"
Oct 30 19:45:48.798580 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.798557    5907 generic.go:356] "Generic (PLEG): container finished" podID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerID="740de951966db6ed02d8d9cf86d300a1468e8033e14afab8f08c503ab737e23c" exitCode=0
Oct 30 19:45:48.798687 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.798640    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" event={"ID":"0832aa0a-f1ee-488f-a44e-4b56726e9b7f","Type":"ContainerDied","Data":"740de951966db6ed02d8d9cf86d300a1468e8033e14afab8f08c503ab737e23c"}
Oct 30 19:45:48.798687 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.798677    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"
Oct 30 19:45:48.798799 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.798696    5907 scope.go:122] "RemoveContainer" containerID="740de951966db6ed02d8d9cf86d300a1468e8033e14afab8f08c503ab737e23c"
Oct 30 19:45:48.798799 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.798681    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" event={"ID":"0832aa0a-f1ee-488f-a44e-4b56726e9b7f","Type":"ContainerDied","Data":"d7a442cbd6e4e78646252416795e2ed819806984c386d7709d8da71489efabe5"}
Oct 30 19:45:48.801576 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.801236    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae7b11b0-xkbwp"
Oct 30 19:45:48.801576 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.801275    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae7b11b0-xkbwp" event={"ID":"3b1170ae-7298-4455-8d90-8e08d345b5a6","Type":"ContainerDied","Data":"1bf9dbee4281826b17e19535ad7f1232c3cd4d9878c787fa8fc8d2b773b86312"}
Oct 30 19:45:48.801576 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.801324    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1bf9dbee4281826b17e19535ad7f1232c3cd4d9878c787fa8fc8d2b773b86312"
Oct 30 19:45:48.812814 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.812797    5907 scope.go:122] "RemoveContainer" containerID="740de951966db6ed02d8d9cf86d300a1468e8033e14afab8f08c503ab737e23c"
Oct 30 19:45:48.813161 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:45:48.813135    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"740de951966db6ed02d8d9cf86d300a1468e8033e14afab8f08c503ab737e23c\": container with ID starting with 740de951966db6ed02d8d9cf86d300a1468e8033e14afab8f08c503ab737e23c not found: ID does not exist" containerID="740de951966db6ed02d8d9cf86d300a1468e8033e14afab8f08c503ab737e23c"
Oct 30 19:45:48.813284 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.813166    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"740de951966db6ed02d8d9cf86d300a1468e8033e14afab8f08c503ab737e23c"} err="failed to get container status \"740de951966db6ed02d8d9cf86d300a1468e8033e14afab8f08c503ab737e23c\": rpc error: code = NotFound desc = could not find container \"740de951966db6ed02d8d9cf86d300a1468e8033e14afab8f08c503ab737e23c\": container with ID starting with 740de951966db6ed02d8d9cf86d300a1468e8033e14afab8f08c503ab737e23c not found: ID does not exist"
Oct 30 19:45:48.858708 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.858680    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2314a32a-7546-45fd-966d-78e512d335f5-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-api-a-3add95-z49q7\" (UID: \"2314a32a-7546-45fd-966d-78e512d335f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3add95-z49q7"
Oct 30 19:45:48.858902 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.858742    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxrjs\" (UniqueName: \"kubernetes.io/projected/2314a32a-7546-45fd-966d-78e512d335f5-kube-api-access-rxrjs\") pod \"maas-api-revoke-keys-e2e-api-a-3add95-z49q7\" (UID: \"2314a32a-7546-45fd-966d-78e512d335f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3add95-z49q7"
Oct 30 19:45:48.858962 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.858930    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2314a32a-7546-45fd-966d-78e512d335f5-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-api-a-3add95-z49q7\" (UID: \"2314a32a-7546-45fd-966d-78e512d335f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3add95-z49q7"
Oct 30 19:45:48.960907 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.960813    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxrjs\" (UniqueName: \"kubernetes.io/projected/2314a32a-7546-45fd-966d-78e512d335f5-kube-api-access-rxrjs\") pod \"maas-api-revoke-keys-e2e-api-a-3add95-z49q7\" (UID: \"2314a32a-7546-45fd-966d-78e512d335f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3add95-z49q7"
Oct 30 19:45:48.961072 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.960944    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2314a32a-7546-45fd-966d-78e512d335f5-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-api-a-3add95-z49q7\" (UID: \"2314a32a-7546-45fd-966d-78e512d335f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3add95-z49q7"
Oct 30 19:45:48.961244 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.961217    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2314a32a-7546-45fd-966d-78e512d335f5-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-api-a-3add95-z49q7\" (UID: \"2314a32a-7546-45fd-966d-78e512d335f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3add95-z49q7"
Oct 30 19:45:48.961713 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.961690    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2314a32a-7546-45fd-966d-78e512d335f5-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-api-a-3add95-z49q7\" (UID: \"2314a32a-7546-45fd-966d-78e512d335f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3add95-z49q7"
Oct 30 19:45:48.963827 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.963802    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2314a32a-7546-45fd-966d-78e512d335f5-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-api-a-3add95-z49q7\" (UID: \"2314a32a-7546-45fd-966d-78e512d335f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3add95-z49q7"
Oct 30 19:45:48.984760 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:48.984689    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxrjs\" (UniqueName: \"kubernetes.io/projected/2314a32a-7546-45fd-966d-78e512d335f5-kube-api-access-rxrjs\") pod \"maas-api-revoke-keys-e2e-api-a-3add95-z49q7\" (UID: \"2314a32a-7546-45fd-966d-78e512d335f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3add95-z49q7"
Oct 30 19:45:49.036740 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:49.036703    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"]
Oct 30 19:45:49.044203 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:49.044173    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7"]
Oct 30 19:45:49.059418 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:49.059392    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3add95-z49q7"
Oct 30 19:45:49.352090 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:49.351990    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3735b2/premium-w2-3735b2-kserve-89b655654-2vsn7" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 19:45:49.389532 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:45:49.389491    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2314a32a_7546_45fd_966d_78e512d335f5.slice/crio-01a1db3c1e2662af023ff1329f74cf31eaaec3a6a78aacd3e231a5e8725526b5 WatchSource:0}: Error finding container 01a1db3c1e2662af023ff1329f74cf31eaaec3a6a78aacd3e231a5e8725526b5: Status 404 returned error can't find the container with id 01a1db3c1e2662af023ff1329f74cf31eaaec3a6a78aacd3e231a5e8725526b5
Oct 30 19:45:49.389727 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:49.389701    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3add95-z49q7"]
Oct 30 19:45:49.810127 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:49.810089    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3add95-z49q7" event={"ID":"2314a32a-7546-45fd-966d-78e512d335f5","Type":"ContainerStarted","Data":"01a1db3c1e2662af023ff1329f74cf31eaaec3a6a78aacd3e231a5e8725526b5"}
Oct 30 19:45:50.290555 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:50.290516    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0832aa0a-f1ee-488f-a44e-4b56726e9b7f" path="/var/lib/kubelet/pods/0832aa0a-f1ee-488f-a44e-4b56726e9b7f/volumes"
Oct 30 19:45:50.821737 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:50.821706    5907 generic.go:356] "Generic (PLEG): container finished" podID="2314a32a-7546-45fd-966d-78e512d335f5" containerID="7f8e72dbac689c04892c385ba54c4e3e7e53c52d28d39d917d6fd90ef0ae0452" exitCode=0
Oct 30 19:45:50.821956 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:50.821804    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3add95-z49q7" event={"ID":"2314a32a-7546-45fd-966d-78e512d335f5","Type":"ContainerDied","Data":"7f8e72dbac689c04892c385ba54c4e3e7e53c52d28d39d917d6fd90ef0ae0452"}
Oct 30 19:45:51.999794 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:51.999770    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3add95-z49q7"
Oct 30 19:45:52.120954 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:52.120922    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2314a32a-7546-45fd-966d-78e512d335f5-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2314a32a-7546-45fd-966d-78e512d335f5-maas-api-openshift-ca\") pod \"2314a32a-7546-45fd-966d-78e512d335f5\" (UID: \"2314a32a-7546-45fd-966d-78e512d335f5\") "
Oct 30 19:45:52.121167 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:52.121036    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2314a32a-7546-45fd-966d-78e512d335f5-kube-api-access-rxrjs\" (UniqueName: \"kubernetes.io/projected/2314a32a-7546-45fd-966d-78e512d335f5-kube-api-access-rxrjs\") pod \"2314a32a-7546-45fd-966d-78e512d335f5\" (UID: \"2314a32a-7546-45fd-966d-78e512d335f5\") "
Oct 30 19:45:52.121167 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:52.121121    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2314a32a-7546-45fd-966d-78e512d335f5-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2314a32a-7546-45fd-966d-78e512d335f5-maas-api-xks-ca\") pod \"2314a32a-7546-45fd-966d-78e512d335f5\" (UID: \"2314a32a-7546-45fd-966d-78e512d335f5\") "
Oct 30 19:45:52.121353 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:52.121320    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2314a32a-7546-45fd-966d-78e512d335f5-maas-api-openshift-ca" pod "2314a32a-7546-45fd-966d-78e512d335f5" (UID: "2314a32a-7546-45fd-966d-78e512d335f5"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:52.121755 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:52.121738    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2314a32a-7546-45fd-966d-78e512d335f5-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:52.123381 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:52.123352    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2314a32a-7546-45fd-966d-78e512d335f5-kube-api-access-rxrjs" pod "2314a32a-7546-45fd-966d-78e512d335f5" (UID: "2314a32a-7546-45fd-966d-78e512d335f5"). InnerVolumeSpecName "kube-api-access-rxrjs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:52.123381 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:52.123371    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2314a32a-7546-45fd-966d-78e512d335f5-maas-api-xks-ca" pod "2314a32a-7546-45fd-966d-78e512d335f5" (UID: "2314a32a-7546-45fd-966d-78e512d335f5"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:45:52.226826 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:52.226786    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxrjs\" (UniqueName: \"kubernetes.io/projected/2314a32a-7546-45fd-966d-78e512d335f5-kube-api-access-rxrjs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:52.226826 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:52.226827    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2314a32a-7546-45fd-966d-78e512d335f5-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:52.463132 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:52.463044    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"]
Oct 30 19:45:52.463314 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:52.463237    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4" podUID="320e693e-a39d-4e8b-9772-cc8e811acc69" containerName="istio-proxy" containerID="cri-o://ba19f803954b374f52aded5cc438d5a6c9ebec14088c75e0ef99dc5ab424185c" gracePeriod=30
Oct 30 19:45:52.841253 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:52.841172    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3add95-z49q7"
Oct 30 19:45:52.841253 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:52.841194    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3add95-z49q7" event={"ID":"2314a32a-7546-45fd-966d-78e512d335f5","Type":"ContainerDied","Data":"01a1db3c1e2662af023ff1329f74cf31eaaec3a6a78aacd3e231a5e8725526b5"}
Oct 30 19:45:52.841253 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:52.841251    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="01a1db3c1e2662af023ff1329f74cf31eaaec3a6a78aacd3e231a5e8725526b5"
Oct 30 19:45:53.264070 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:53.264039    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ef9a5a-wxc7j"]
Oct 30 19:45:53.264826 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:53.264812    5907 cpu_manager.go:414] "containerMap: removing container" podUID="2314a32a-7546-45fd-966d-78e512d335f5" containerName="revoke-keys"
Oct 30 19:45:53.264910 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:53.264827    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2314a32a-7546-45fd-966d-78e512d335f5" containerName="revoke-keys"
Oct 30 19:45:53.265004 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:53.264992    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="2314a32a-7546-45fd-966d-78e512d335f5" containerName="revoke-keys"
Oct 30 19:45:53.269002 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:53.268985    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ef9a5a-wxc7j"
Oct 30 19:45:53.294738 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:53.294699    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ef9a5a-wxc7j"]
Oct 30 19:45:53.295296 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:53.295274    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:45:53.296293 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:53.296270    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:45:53.300033 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:53.300011    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ns4q\" (UniqueName: \"kubernetes.io/projected/810e8003-26f9-422c-a7e8-9cb10c76ce45-kube-api-access-8ns4q\") pod \"maas-api-revoke-keys-e2e-worker-w6-ef9a5a-wxc7j\" (UID: \"810e8003-26f9-422c-a7e8-9cb10c76ce45\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ef9a5a-wxc7j"
Oct 30 19:45:53.300325 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:53.300303    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/810e8003-26f9-422c-a7e8-9cb10c76ce45-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-ef9a5a-wxc7j\" (UID: \"810e8003-26f9-422c-a7e8-9cb10c76ce45\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ef9a5a-wxc7j"
Oct 30 19:45:53.300413 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:53.300395    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/810e8003-26f9-422c-a7e8-9cb10c76ce45-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-ef9a5a-wxc7j\" (UID: \"810e8003-26f9-422c-a7e8-9cb10c76ce45\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ef9a5a-wxc7j"
Oct 30 19:45:53.401446 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:53.401411    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/810e8003-26f9-422c-a7e8-9cb10c76ce45-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-ef9a5a-wxc7j\" (UID: \"810e8003-26f9-422c-a7e8-9cb10c76ce45\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ef9a5a-wxc7j"
Oct 30 19:45:53.401692 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:53.401488    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/810e8003-26f9-422c-a7e8-9cb10c76ce45-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-ef9a5a-wxc7j\" (UID: \"810e8003-26f9-422c-a7e8-9cb10c76ce45\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ef9a5a-wxc7j"
Oct 30 19:45:53.401692 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:53.401551    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ns4q\" (UniqueName: \"kubernetes.io/projected/810e8003-26f9-422c-a7e8-9cb10c76ce45-kube-api-access-8ns4q\") pod \"maas-api-revoke-keys-e2e-worker-w6-ef9a5a-wxc7j\" (UID: \"810e8003-26f9-422c-a7e8-9cb10c76ce45\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ef9a5a-wxc7j"
Oct 30 19:45:53.402265 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:53.402240    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/810e8003-26f9-422c-a7e8-9cb10c76ce45-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-ef9a5a-wxc7j\" (UID: \"810e8003-26f9-422c-a7e8-9cb10c76ce45\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ef9a5a-wxc7j"
Oct 30 19:45:53.404130 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:53.404109    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/810e8003-26f9-422c-a7e8-9cb10c76ce45-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-ef9a5a-wxc7j\" (UID: \"810e8003-26f9-422c-a7e8-9cb10c76ce45\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ef9a5a-wxc7j"
Oct 30 19:45:53.425548 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:53.425516    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ns4q\" (UniqueName: \"kubernetes.io/projected/810e8003-26f9-422c-a7e8-9cb10c76ce45-kube-api-access-8ns4q\") pod \"maas-api-revoke-keys-e2e-worker-w6-ef9a5a-wxc7j\" (UID: \"810e8003-26f9-422c-a7e8-9cb10c76ce45\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ef9a5a-wxc7j"
Oct 30 19:45:53.584496 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:53.584421    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ef9a5a-wxc7j"
Oct 30 19:45:53.899585 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:53.899554    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ef9a5a-wxc7j"]
Oct 30 19:45:53.900042 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:45:53.900005    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod810e8003_26f9_422c_a7e8_9cb10c76ce45.slice/crio-bf8c033a6dcff3b8fd190e7eedeb7d13493f9f8645c35341c71e13b4b4f89efe WatchSource:0}: Error finding container bf8c033a6dcff3b8fd190e7eedeb7d13493f9f8645c35341c71e13b4b4f89efe: Status 404 returned error can't find the container with id bf8c033a6dcff3b8fd190e7eedeb7d13493f9f8645c35341c71e13b4b4f89efe
Oct 30 19:45:54.860842 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:54.860808    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ef9a5a-wxc7j" event={"ID":"810e8003-26f9-422c-a7e8-9cb10c76ce45","Type":"ContainerStarted","Data":"8954d56c4cce96b4f6365487cebef3486212fa77b07ee0ecd9683d879f70ff8a"}
Oct 30 19:45:54.861349 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:54.860882    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ef9a5a-wxc7j" event={"ID":"810e8003-26f9-422c-a7e8-9cb10c76ce45","Type":"ContainerStarted","Data":"bf8c033a6dcff3b8fd190e7eedeb7d13493f9f8645c35341c71e13b4b4f89efe"}
Oct 30 19:45:54.952461 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:54.952414    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ef9a5a-wxc7j" podStartSLOduration=1.952399759 podStartE2EDuration="1.952399759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:54.950115441 +0000 UTC m=+1466.793047379" watchObservedRunningTime="2026-10-30 19:45:54.952399759 +0000 UTC m=+1466.795331722"
Oct 30 19:45:55.870523 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:55.870482    5907 generic.go:356] "Generic (PLEG): container finished" podID="810e8003-26f9-422c-a7e8-9cb10c76ce45" containerID="8954d56c4cce96b4f6365487cebef3486212fa77b07ee0ecd9683d879f70ff8a" exitCode=0
Oct 30 19:45:55.870988 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:55.870558    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ef9a5a-wxc7j" event={"ID":"810e8003-26f9-422c-a7e8-9cb10c76ce45","Type":"ContainerDied","Data":"8954d56c4cce96b4f6365487cebef3486212fa77b07ee0ecd9683d879f70ff8a"}
Oct 30 19:45:56.477938 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:56.477904    5907 patch_prober.go:29] interesting pod/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:45:56.478106 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:56.477961    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4" podUID="320e693e-a39d-4e8b-9772-cc8e811acc69" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:45:57.039282 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.039251    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ef9a5a-wxc7j"
Oct 30 19:45:57.170044 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.169960    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/810e8003-26f9-422c-a7e8-9cb10c76ce45-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/810e8003-26f9-422c-a7e8-9cb10c76ce45-maas-api-xks-ca\") pod \"810e8003-26f9-422c-a7e8-9cb10c76ce45\" (UID: \"810e8003-26f9-422c-a7e8-9cb10c76ce45\") "
Oct 30 19:45:57.170044 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.170018    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/810e8003-26f9-422c-a7e8-9cb10c76ce45-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/810e8003-26f9-422c-a7e8-9cb10c76ce45-maas-api-openshift-ca\") pod \"810e8003-26f9-422c-a7e8-9cb10c76ce45\" (UID: \"810e8003-26f9-422c-a7e8-9cb10c76ce45\") "
Oct 30 19:45:57.170389 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.170364    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/810e8003-26f9-422c-a7e8-9cb10c76ce45-maas-api-openshift-ca" pod "810e8003-26f9-422c-a7e8-9cb10c76ce45" (UID: "810e8003-26f9-422c-a7e8-9cb10c76ce45"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:57.170446 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.170376    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/810e8003-26f9-422c-a7e8-9cb10c76ce45-kube-api-access-8ns4q\" (UniqueName: \"kubernetes.io/projected/810e8003-26f9-422c-a7e8-9cb10c76ce45-kube-api-access-8ns4q\") pod \"810e8003-26f9-422c-a7e8-9cb10c76ce45\" (UID: \"810e8003-26f9-422c-a7e8-9cb10c76ce45\") "
Oct 30 19:45:57.171042 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.170973    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/810e8003-26f9-422c-a7e8-9cb10c76ce45-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:57.172477 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.172449    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/810e8003-26f9-422c-a7e8-9cb10c76ce45-maas-api-xks-ca" pod "810e8003-26f9-422c-a7e8-9cb10c76ce45" (UID: "810e8003-26f9-422c-a7e8-9cb10c76ce45"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:45:57.172554 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.172488    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/810e8003-26f9-422c-a7e8-9cb10c76ce45-kube-api-access-8ns4q" pod "810e8003-26f9-422c-a7e8-9cb10c76ce45" (UID: "810e8003-26f9-422c-a7e8-9cb10c76ce45"). InnerVolumeSpecName "kube-api-access-8ns4q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:57.272583 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.272548    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ns4q\" (UniqueName: \"kubernetes.io/projected/810e8003-26f9-422c-a7e8-9cb10c76ce45-kube-api-access-8ns4q\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:57.272583 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.272580    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/810e8003-26f9-422c-a7e8-9cb10c76ce45-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:57.747117 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.747091    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:45:57.750813 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.750787    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-e0e300-7b97d947b4-h2v9z"]
Oct 30 19:45:57.783238 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.783210    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-e0e300-589877f74c-xnhdg"]
Oct 30 19:45:57.900320 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.892530    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/320e693e-a39d-4e8b-9772-cc8e811acc69-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/320e693e-a39d-4e8b-9772-cc8e811acc69-istio-data\") pod \"320e693e-a39d-4e8b-9772-cc8e811acc69\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") "
Oct 30 19:45:57.900320 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.892603    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/320e693e-a39d-4e8b-9772-cc8e811acc69-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/320e693e-a39d-4e8b-9772-cc8e811acc69-workload-certs\") pod \"320e693e-a39d-4e8b-9772-cc8e811acc69\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") "
Oct 30 19:45:57.900320 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.892646    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/320e693e-a39d-4e8b-9772-cc8e811acc69-istio-token\" (UniqueName: \"kubernetes.io/projected/320e693e-a39d-4e8b-9772-cc8e811acc69-istio-token\") pod \"320e693e-a39d-4e8b-9772-cc8e811acc69\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") "
Oct 30 19:45:57.900320 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.892692    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/320e693e-a39d-4e8b-9772-cc8e811acc69-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/320e693e-a39d-4e8b-9772-cc8e811acc69-workload-socket\") pod \"320e693e-a39d-4e8b-9772-cc8e811acc69\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") "
Oct 30 19:45:57.900320 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.892762    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/320e693e-a39d-4e8b-9772-cc8e811acc69-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/320e693e-a39d-4e8b-9772-cc8e811acc69-istio-envoy\") pod \"320e693e-a39d-4e8b-9772-cc8e811acc69\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") "
Oct 30 19:45:57.900320 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.897328    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/320e693e-a39d-4e8b-9772-cc8e811acc69-kube-api-access-whs48" pod "320e693e-a39d-4e8b-9772-cc8e811acc69" (UID: "320e693e-a39d-4e8b-9772-cc8e811acc69"). InnerVolumeSpecName "kube-api-access-whs48". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:57.900320 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.897623    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/320e693e-a39d-4e8b-9772-cc8e811acc69-istio-data" pod "320e693e-a39d-4e8b-9772-cc8e811acc69" (UID: "320e693e-a39d-4e8b-9772-cc8e811acc69"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:57.900320 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.897945    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/320e693e-a39d-4e8b-9772-cc8e811acc69-workload-certs" pod "320e693e-a39d-4e8b-9772-cc8e811acc69" (UID: "320e693e-a39d-4e8b-9772-cc8e811acc69"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:57.900320 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.897986    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/320e693e-a39d-4e8b-9772-cc8e811acc69-kube-api-access-whs48\" (UniqueName: \"kubernetes.io/projected/320e693e-a39d-4e8b-9772-cc8e811acc69-kube-api-access-whs48\") pod \"320e693e-a39d-4e8b-9772-cc8e811acc69\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") "
Oct 30 19:45:57.900320 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.898081    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/320e693e-a39d-4e8b-9772-cc8e811acc69-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/320e693e-a39d-4e8b-9772-cc8e811acc69-credential-socket\") pod \"320e693e-a39d-4e8b-9772-cc8e811acc69\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") "
Oct 30 19:45:57.900320 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.898174    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/320e693e-a39d-4e8b-9772-cc8e811acc69-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/320e693e-a39d-4e8b-9772-cc8e811acc69-istio-podinfo\") pod \"320e693e-a39d-4e8b-9772-cc8e811acc69\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") "
Oct 30 19:45:57.900320 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.898245    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/320e693e-a39d-4e8b-9772-cc8e811acc69-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/320e693e-a39d-4e8b-9772-cc8e811acc69-istiod-ca-cert\") pod \"320e693e-a39d-4e8b-9772-cc8e811acc69\" (UID: \"320e693e-a39d-4e8b-9772-cc8e811acc69\") "
Oct 30 19:45:57.900320 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.899057    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/320e693e-a39d-4e8b-9772-cc8e811acc69-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:57.900320 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.899081    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/320e693e-a39d-4e8b-9772-cc8e811acc69-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:57.900320 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.899095    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whs48\" (UniqueName: \"kubernetes.io/projected/320e693e-a39d-4e8b-9772-cc8e811acc69-kube-api-access-whs48\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:57.900320 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.899402    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/320e693e-a39d-4e8b-9772-cc8e811acc69-istiod-ca-cert" pod "320e693e-a39d-4e8b-9772-cc8e811acc69" (UID: "320e693e-a39d-4e8b-9772-cc8e811acc69"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:57.902024 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.901906    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/320e693e-a39d-4e8b-9772-cc8e811acc69-istio-token" pod "320e693e-a39d-4e8b-9772-cc8e811acc69" (UID: "320e693e-a39d-4e8b-9772-cc8e811acc69"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:57.902204 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.902133    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/320e693e-a39d-4e8b-9772-cc8e811acc69-credential-socket" pod "320e693e-a39d-4e8b-9772-cc8e811acc69" (UID: "320e693e-a39d-4e8b-9772-cc8e811acc69"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:57.903321 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.903282    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/320e693e-a39d-4e8b-9772-cc8e811acc69-workload-socket" pod "320e693e-a39d-4e8b-9772-cc8e811acc69" (UID: "320e693e-a39d-4e8b-9772-cc8e811acc69"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:57.906898 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.904763    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/320e693e-a39d-4e8b-9772-cc8e811acc69-istio-envoy" pod "320e693e-a39d-4e8b-9772-cc8e811acc69" (UID: "320e693e-a39d-4e8b-9772-cc8e811acc69"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:57.906898 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.906409    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/320e693e-a39d-4e8b-9772-cc8e811acc69-istio-podinfo" pod "320e693e-a39d-4e8b-9772-cc8e811acc69" (UID: "320e693e-a39d-4e8b-9772-cc8e811acc69"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:45:57.911954 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.911936    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ef9a5a-wxc7j"
Oct 30 19:45:57.912056 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.911975    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ef9a5a-wxc7j" event={"ID":"810e8003-26f9-422c-a7e8-9cb10c76ce45","Type":"ContainerDied","Data":"bf8c033a6dcff3b8fd190e7eedeb7d13493f9f8645c35341c71e13b4b4f89efe"}
Oct 30 19:45:57.912056 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.912028    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bf8c033a6dcff3b8fd190e7eedeb7d13493f9f8645c35341c71e13b4b4f89efe"
Oct 30 19:45:57.914097 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.914070    5907 generic.go:356] "Generic (PLEG): container finished" podID="320e693e-a39d-4e8b-9772-cc8e811acc69" containerID="ba19f803954b374f52aded5cc438d5a6c9ebec14088c75e0ef99dc5ab424185c" exitCode=0
Oct 30 19:45:57.914186 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.914109    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"
Oct 30 19:45:57.914186 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.914143    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4" event={"ID":"320e693e-a39d-4e8b-9772-cc8e811acc69","Type":"ContainerDied","Data":"ba19f803954b374f52aded5cc438d5a6c9ebec14088c75e0ef99dc5ab424185c"}
Oct 30 19:45:57.914272 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.914189    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4" event={"ID":"320e693e-a39d-4e8b-9772-cc8e811acc69","Type":"ContainerDied","Data":"67242aeccfb3501e19b0a4d3a0a5e7d674b07a1d9f2a64faa9b7284fc14d396e"}
Oct 30 19:45:57.914272 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.914221    5907 scope.go:122] "RemoveContainer" containerID="ba19f803954b374f52aded5cc438d5a6c9ebec14088c75e0ef99dc5ab424185c"
Oct 30 19:45:57.927670 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.927653    5907 scope.go:122] "RemoveContainer" containerID="12edb43d3409e56c27f3fc6e214c9f1184daea65e1a9bdf5f804d720eb82ece0"
Oct 30 19:45:57.941655 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.941632    5907 scope.go:122] "RemoveContainer" containerID="ba19f803954b374f52aded5cc438d5a6c9ebec14088c75e0ef99dc5ab424185c"
Oct 30 19:45:57.942140 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:45:57.942114    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba19f803954b374f52aded5cc438d5a6c9ebec14088c75e0ef99dc5ab424185c\": container with ID starting with ba19f803954b374f52aded5cc438d5a6c9ebec14088c75e0ef99dc5ab424185c not found: ID does not exist" containerID="ba19f803954b374f52aded5cc438d5a6c9ebec14088c75e0ef99dc5ab424185c"
Oct 30 19:45:57.942248 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.942155    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba19f803954b374f52aded5cc438d5a6c9ebec14088c75e0ef99dc5ab424185c"} err="failed to get container status \"ba19f803954b374f52aded5cc438d5a6c9ebec14088c75e0ef99dc5ab424185c\": rpc error: code = NotFound desc = could not find container \"ba19f803954b374f52aded5cc438d5a6c9ebec14088c75e0ef99dc5ab424185c\": container with ID starting with ba19f803954b374f52aded5cc438d5a6c9ebec14088c75e0ef99dc5ab424185c not found: ID does not exist"
Oct 30 19:45:57.942248 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.942180    5907 scope.go:122] "RemoveContainer" containerID="12edb43d3409e56c27f3fc6e214c9f1184daea65e1a9bdf5f804d720eb82ece0"
Oct 30 19:45:57.942596 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:45:57.942574    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12edb43d3409e56c27f3fc6e214c9f1184daea65e1a9bdf5f804d720eb82ece0\": container with ID starting with 12edb43d3409e56c27f3fc6e214c9f1184daea65e1a9bdf5f804d720eb82ece0 not found: ID does not exist" containerID="12edb43d3409e56c27f3fc6e214c9f1184daea65e1a9bdf5f804d720eb82ece0"
Oct 30 19:45:57.942670 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.942607    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12edb43d3409e56c27f3fc6e214c9f1184daea65e1a9bdf5f804d720eb82ece0"} err="failed to get container status \"12edb43d3409e56c27f3fc6e214c9f1184daea65e1a9bdf5f804d720eb82ece0\": rpc error: code = NotFound desc = could not find container \"12edb43d3409e56c27f3fc6e214c9f1184daea65e1a9bdf5f804d720eb82ece0\": container with ID starting with 12edb43d3409e56c27f3fc6e214c9f1184daea65e1a9bdf5f804d720eb82ece0 not found: ID does not exist"
Oct 30 19:45:57.979093 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.979057    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"]
Oct 30 19:45:57.982874 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:57.982774    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-086f2a-openshift-default-5d4d99c7c-b2kv4"]
Oct 30 19:45:58.001082 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:58.000639    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/320e693e-a39d-4e8b-9772-cc8e811acc69-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:58.001082 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:58.000676    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/320e693e-a39d-4e8b-9772-cc8e811acc69-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:58.001082 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:58.000691    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/320e693e-a39d-4e8b-9772-cc8e811acc69-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:58.001082 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:58.000707    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/320e693e-a39d-4e8b-9772-cc8e811acc69-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:58.001082 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:58.000723    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/320e693e-a39d-4e8b-9772-cc8e811acc69-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:58.001082 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:58.000736    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/320e693e-a39d-4e8b-9772-cc8e811acc69-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:45:58.049899 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:58.041653    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-3735b2-795c89ffc9-rtdm4" podUID="97e31b29-cb25-492c-bd66-fcab4b422ab9" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:45:58.049899 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:58.043231    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-3735b2-6f754b9b8b-b9rgh" podUID="b1714315-7df0-4860-8759-bdfab7c13824" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:45:58.077342 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:58.077305    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"]
Oct 30 19:45:58.290335 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:58.290233    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="320e693e-a39d-4e8b-9772-cc8e811acc69" path="/var/lib/kubelet/pods/320e693e-a39d-4e8b-9772-cc8e811acc69/volumes"
Oct 30 19:45:59.181976 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:59.181940    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-3add95-5f665b5b94-hdzl8"]
Oct 30 19:45:59.223540 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:45:59.223509    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-3add95-d9bb45d9f-sjmpj"]
Oct 30 19:46:02.759888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:02.752166    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-e0e300-7b97d947b4-h2v9z" podUID="23069174-9b21-412a-aa6c-118d8446da71" containerName="payload-processing" containerID="cri-o://88adf0683bb6ad8f371b847263c16d4c96fc6a482e8db335f260184a072754c0" gracePeriod=25
Oct 30 19:46:02.783788 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:02.783750    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-e0e300-589877f74c-xnhdg" podUID="63fc1ff9-fc06-4c20-934f-3667809c844e" containerName="payload-pre-processing" containerID="cri-o://438c1a697da47b4710c7bd61c288a453f48bd869bbc92a4fe4e7f2703c1ba4af" gracePeriod=25
Oct 30 19:46:03.068014 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:03.067978    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-e0e300-7b97d947b4-h2v9z"
Oct 30 19:46:03.069992 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:03.069961    5907 generic.go:356] "Generic (PLEG): container finished" podID="63fc1ff9-fc06-4c20-934f-3667809c844e" containerID="438c1a697da47b4710c7bd61c288a453f48bd869bbc92a4fe4e7f2703c1ba4af" exitCode=0
Oct 30 19:46:03.070153 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:03.070039    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-e0e300-589877f74c-xnhdg" event={"ID":"63fc1ff9-fc06-4c20-934f-3667809c844e","Type":"ContainerDied","Data":"438c1a697da47b4710c7bd61c288a453f48bd869bbc92a4fe4e7f2703c1ba4af"}
Oct 30 19:46:03.071832 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:03.071809    5907 generic.go:356] "Generic (PLEG): container finished" podID="23069174-9b21-412a-aa6c-118d8446da71" containerID="88adf0683bb6ad8f371b847263c16d4c96fc6a482e8db335f260184a072754c0" exitCode=0
Oct 30 19:46:03.072013 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:03.071851    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-e0e300-7b97d947b4-h2v9z"
Oct 30 19:46:03.072013 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:03.071967    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-e0e300-7b97d947b4-h2v9z" event={"ID":"23069174-9b21-412a-aa6c-118d8446da71","Type":"ContainerDied","Data":"88adf0683bb6ad8f371b847263c16d4c96fc6a482e8db335f260184a072754c0"}
Oct 30 19:46:03.072013 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:03.072004    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-e0e300-7b97d947b4-h2v9z" event={"ID":"23069174-9b21-412a-aa6c-118d8446da71","Type":"ContainerDied","Data":"1b95a872787651cc37b8b8dc1008e9ab8c3212e4adc0e49b0540bdb46851fb3d"}
Oct 30 19:46:03.072203 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:03.072033    5907 scope.go:122] "RemoveContainer" containerID="88adf0683bb6ad8f371b847263c16d4c96fc6a482e8db335f260184a072754c0"
Oct 30 19:46:03.085266 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:03.085247    5907 scope.go:122] "RemoveContainer" containerID="88adf0683bb6ad8f371b847263c16d4c96fc6a482e8db335f260184a072754c0"
Oct 30 19:46:03.085594 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:46:03.085573    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88adf0683bb6ad8f371b847263c16d4c96fc6a482e8db335f260184a072754c0\": container with ID starting with 88adf0683bb6ad8f371b847263c16d4c96fc6a482e8db335f260184a072754c0 not found: ID does not exist" containerID="88adf0683bb6ad8f371b847263c16d4c96fc6a482e8db335f260184a072754c0"
Oct 30 19:46:03.085699 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:03.085599    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88adf0683bb6ad8f371b847263c16d4c96fc6a482e8db335f260184a072754c0"} err="failed to get container status \"88adf0683bb6ad8f371b847263c16d4c96fc6a482e8db335f260184a072754c0\": rpc error: code = NotFound desc = could not find container \"88adf0683bb6ad8f371b847263c16d4c96fc6a482e8db335f260184a072754c0\": container with ID starting with 88adf0683bb6ad8f371b847263c16d4c96fc6a482e8db335f260184a072754c0 not found: ID does not exist"
Oct 30 19:46:03.120328 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:03.120302    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-e0e300-589877f74c-xnhdg"
Oct 30 19:46:03.189267 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:03.189235    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/23069174-9b21-412a-aa6c-118d8446da71-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23069174-9b21-412a-aa6c-118d8446da71-plugins-config-volume\") pod \"23069174-9b21-412a-aa6c-118d8446da71\" (UID: \"23069174-9b21-412a-aa6c-118d8446da71\") "
Oct 30 19:46:03.189466 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:03.189366    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/23069174-9b21-412a-aa6c-118d8446da71-kube-api-access-spgzf\" (UniqueName: \"kubernetes.io/projected/23069174-9b21-412a-aa6c-118d8446da71-kube-api-access-spgzf\") pod \"23069174-9b21-412a-aa6c-118d8446da71\" (UID: \"23069174-9b21-412a-aa6c-118d8446da71\") "
Oct 30 19:46:03.189640 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:03.189605    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23069174-9b21-412a-aa6c-118d8446da71-plugins-config-volume" pod "23069174-9b21-412a-aa6c-118d8446da71" (UID: "23069174-9b21-412a-aa6c-118d8446da71"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:03.190076 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:03.190046    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23069174-9b21-412a-aa6c-118d8446da71-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:03.192052 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:03.192026    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23069174-9b21-412a-aa6c-118d8446da71-kube-api-access-spgzf" pod "23069174-9b21-412a-aa6c-118d8446da71" (UID: "23069174-9b21-412a-aa6c-118d8446da71"). InnerVolumeSpecName "kube-api-access-spgzf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:03.291290 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:03.291208    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/63fc1ff9-fc06-4c20-934f-3667809c844e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63fc1ff9-fc06-4c20-934f-3667809c844e-plugins-config-volume\") pod \"63fc1ff9-fc06-4c20-934f-3667809c844e\" (UID: \"63fc1ff9-fc06-4c20-934f-3667809c844e\") "
Oct 30 19:46:03.291441 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:03.291321    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/63fc1ff9-fc06-4c20-934f-3667809c844e-kube-api-access-wmv2l\" (UniqueName: \"kubernetes.io/projected/63fc1ff9-fc06-4c20-934f-3667809c844e-kube-api-access-wmv2l\") pod \"63fc1ff9-fc06-4c20-934f-3667809c844e\" (UID: \"63fc1ff9-fc06-4c20-934f-3667809c844e\") "
Oct 30 19:46:03.291596 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:03.291572    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63fc1ff9-fc06-4c20-934f-3667809c844e-plugins-config-volume" pod "63fc1ff9-fc06-4c20-934f-3667809c844e" (UID: "63fc1ff9-fc06-4c20-934f-3667809c844e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:03.291771 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:03.291752    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63fc1ff9-fc06-4c20-934f-3667809c844e-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:03.291771 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:03.291770    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spgzf\" (UniqueName: \"kubernetes.io/projected/23069174-9b21-412a-aa6c-118d8446da71-kube-api-access-spgzf\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:03.293518 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:03.293489    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63fc1ff9-fc06-4c20-934f-3667809c844e-kube-api-access-wmv2l" pod "63fc1ff9-fc06-4c20-934f-3667809c844e" (UID: "63fc1ff9-fc06-4c20-934f-3667809c844e"). InnerVolumeSpecName "kube-api-access-wmv2l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:03.392894 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:03.392850    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmv2l\" (UniqueName: \"kubernetes.io/projected/63fc1ff9-fc06-4c20-934f-3667809c844e-kube-api-access-wmv2l\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:03.433442 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:03.433411    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-e0e300-7b97d947b4-h2v9z"]
Oct 30 19:46:03.443428 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:03.443403    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-e0e300-7b97d947b4-h2v9z"]
Oct 30 19:46:03.799468 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:03.799429    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-ef9a5a-59fc6bf976-zfzks"]
Oct 30 19:46:03.859850 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:03.858979    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-ef9a5a-796b5cf9dc-kmcg7"]
Oct 30 19:46:03.961892 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:03.961841    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-3735b2-6f754b9b8b-b9rgh"
Oct 30 19:46:04.002134 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.002107    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-3735b2-795c89ffc9-rtdm4"
Oct 30 19:46:04.082830 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.082771    5907 generic.go:356] "Generic (PLEG): container finished" podID="97e31b29-cb25-492c-bd66-fcab4b422ab9" containerID="69a5752e0cc2ce257d207fec4bb9b500ae53910687a54933a950146d925642e3" exitCode=0
Oct 30 19:46:04.083062 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.082832    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-3735b2-795c89ffc9-rtdm4"
Oct 30 19:46:04.083062 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.082874    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-3735b2-795c89ffc9-rtdm4" event={"ID":"97e31b29-cb25-492c-bd66-fcab4b422ab9","Type":"ContainerDied","Data":"69a5752e0cc2ce257d207fec4bb9b500ae53910687a54933a950146d925642e3"}
Oct 30 19:46:04.083062 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.082992    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-3735b2-795c89ffc9-rtdm4" event={"ID":"97e31b29-cb25-492c-bd66-fcab4b422ab9","Type":"ContainerDied","Data":"1a238b64f3e47159b3c1f69f2fbdc948da75be16aebccb18028ea527eef2efa5"}
Oct 30 19:46:04.083062 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.083045    5907 scope.go:122] "RemoveContainer" containerID="69a5752e0cc2ce257d207fec4bb9b500ae53910687a54933a950146d925642e3"
Oct 30 19:46:04.084932 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.084911    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-e0e300-589877f74c-xnhdg" event={"ID":"63fc1ff9-fc06-4c20-934f-3667809c844e","Type":"ContainerDied","Data":"6f5a095799fff45c12b39f1e10166672229d8e8dd27e1b727bd64b76e436132f"}
Oct 30 19:46:04.084932 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.084928    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-e0e300-589877f74c-xnhdg"
Oct 30 19:46:04.087979 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.087954    5907 generic.go:356] "Generic (PLEG): container finished" podID="b1714315-7df0-4860-8759-bdfab7c13824" containerID="4997148dc75040d79541acda6e57517025cdce6a421afddc7216ba87cb6f8b43" exitCode=0
Oct 30 19:46:04.088088 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.088060    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-3735b2-6f754b9b8b-b9rgh" event={"ID":"b1714315-7df0-4860-8759-bdfab7c13824","Type":"ContainerDied","Data":"4997148dc75040d79541acda6e57517025cdce6a421afddc7216ba87cb6f8b43"}
Oct 30 19:46:04.088151 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.088098    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-3735b2-6f754b9b8b-b9rgh" event={"ID":"b1714315-7df0-4860-8759-bdfab7c13824","Type":"ContainerDied","Data":"b2c7ecaf9ef6ca845b30ef6f719e980734c4f6a6d2f2eb37762de0c312b3f412"}
Oct 30 19:46:04.088207 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.088162    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-3735b2-6f754b9b8b-b9rgh"
Oct 30 19:46:04.096607 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.096562    5907 scope.go:122] "RemoveContainer" containerID="69a5752e0cc2ce257d207fec4bb9b500ae53910687a54933a950146d925642e3"
Oct 30 19:46:04.096933 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:46:04.096910    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69a5752e0cc2ce257d207fec4bb9b500ae53910687a54933a950146d925642e3\": container with ID starting with 69a5752e0cc2ce257d207fec4bb9b500ae53910687a54933a950146d925642e3 not found: ID does not exist" containerID="69a5752e0cc2ce257d207fec4bb9b500ae53910687a54933a950146d925642e3"
Oct 30 19:46:04.097095 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.096944    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69a5752e0cc2ce257d207fec4bb9b500ae53910687a54933a950146d925642e3"} err="failed to get container status \"69a5752e0cc2ce257d207fec4bb9b500ae53910687a54933a950146d925642e3\": rpc error: code = NotFound desc = could not find container \"69a5752e0cc2ce257d207fec4bb9b500ae53910687a54933a950146d925642e3\": container with ID starting with 69a5752e0cc2ce257d207fec4bb9b500ae53910687a54933a950146d925642e3 not found: ID does not exist"
Oct 30 19:46:04.097095 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.096968    5907 scope.go:122] "RemoveContainer" containerID="438c1a697da47b4710c7bd61c288a453f48bd869bbc92a4fe4e7f2703c1ba4af"
Oct 30 19:46:04.102074 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.102055    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/97e31b29-cb25-492c-bd66-fcab4b422ab9-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97e31b29-cb25-492c-bd66-fcab4b422ab9-plugins-config-volume\") pod \"97e31b29-cb25-492c-bd66-fcab4b422ab9\" (UID: \"97e31b29-cb25-492c-bd66-fcab4b422ab9\") "
Oct 30 19:46:04.102161 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.102099    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/97e31b29-cb25-492c-bd66-fcab4b422ab9-kube-api-access-27bqp\" (UniqueName: \"kubernetes.io/projected/97e31b29-cb25-492c-bd66-fcab4b422ab9-kube-api-access-27bqp\") pod \"97e31b29-cb25-492c-bd66-fcab4b422ab9\" (UID: \"97e31b29-cb25-492c-bd66-fcab4b422ab9\") "
Oct 30 19:46:04.102161 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.102137    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b1714315-7df0-4860-8759-bdfab7c13824-kube-api-access-868pq\" (UniqueName: \"kubernetes.io/projected/b1714315-7df0-4860-8759-bdfab7c13824-kube-api-access-868pq\") pod \"b1714315-7df0-4860-8759-bdfab7c13824\" (UID: \"b1714315-7df0-4860-8759-bdfab7c13824\") "
Oct 30 19:46:04.102289 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.102273    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b1714315-7df0-4860-8759-bdfab7c13824-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1714315-7df0-4860-8759-bdfab7c13824-plugins-config-volume\") pod \"b1714315-7df0-4860-8759-bdfab7c13824\" (UID: \"b1714315-7df0-4860-8759-bdfab7c13824\") "
Oct 30 19:46:04.102507 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.102477    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97e31b29-cb25-492c-bd66-fcab4b422ab9-plugins-config-volume" pod "97e31b29-cb25-492c-bd66-fcab4b422ab9" (UID: "97e31b29-cb25-492c-bd66-fcab4b422ab9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:04.103002 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.102723    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1714315-7df0-4860-8759-bdfab7c13824-plugins-config-volume" pod "b1714315-7df0-4860-8759-bdfab7c13824" (UID: "b1714315-7df0-4860-8759-bdfab7c13824"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:04.103002 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.102925    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97e31b29-cb25-492c-bd66-fcab4b422ab9-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:04.103002 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.102940    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1714315-7df0-4860-8759-bdfab7c13824-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:04.104548 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.104521    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97e31b29-cb25-492c-bd66-fcab4b422ab9-kube-api-access-27bqp" pod "97e31b29-cb25-492c-bd66-fcab4b422ab9" (UID: "97e31b29-cb25-492c-bd66-fcab4b422ab9"). InnerVolumeSpecName "kube-api-access-27bqp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:04.104695 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.104676    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1714315-7df0-4860-8759-bdfab7c13824-kube-api-access-868pq" pod "b1714315-7df0-4860-8759-bdfab7c13824" (UID: "b1714315-7df0-4860-8759-bdfab7c13824"). InnerVolumeSpecName "kube-api-access-868pq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:04.110121 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.110097    5907 scope.go:122] "RemoveContainer" containerID="4997148dc75040d79541acda6e57517025cdce6a421afddc7216ba87cb6f8b43"
Oct 30 19:46:04.124399 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.124377    5907 scope.go:122] "RemoveContainer" containerID="4997148dc75040d79541acda6e57517025cdce6a421afddc7216ba87cb6f8b43"
Oct 30 19:46:04.124795 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:46:04.124770    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4997148dc75040d79541acda6e57517025cdce6a421afddc7216ba87cb6f8b43\": container with ID starting with 4997148dc75040d79541acda6e57517025cdce6a421afddc7216ba87cb6f8b43 not found: ID does not exist" containerID="4997148dc75040d79541acda6e57517025cdce6a421afddc7216ba87cb6f8b43"
Oct 30 19:46:04.124913 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.124808    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4997148dc75040d79541acda6e57517025cdce6a421afddc7216ba87cb6f8b43"} err="failed to get container status \"4997148dc75040d79541acda6e57517025cdce6a421afddc7216ba87cb6f8b43\": rpc error: code = NotFound desc = could not find container \"4997148dc75040d79541acda6e57517025cdce6a421afddc7216ba87cb6f8b43\": container with ID starting with 4997148dc75040d79541acda6e57517025cdce6a421afddc7216ba87cb6f8b43 not found: ID does not exist"
Oct 30 19:46:04.182974 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.182881    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-3add95-5f665b5b94-hdzl8" podUID="7cd1ce0b-3b85-4701-9f18-abdf8a83e564" containerName="payload-processing" containerID="cri-o://d5b866d9b799e30cde92d90bbacd5c90e0524c681da7f081195e7cbd5c7f85c7" gracePeriod=25
Oct 30 19:46:04.204082 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.204051    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-868pq\" (UniqueName: \"kubernetes.io/projected/b1714315-7df0-4860-8759-bdfab7c13824-kube-api-access-868pq\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:04.204082 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.204082    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27bqp\" (UniqueName: \"kubernetes.io/projected/97e31b29-cb25-492c-bd66-fcab4b422ab9-kube-api-access-27bqp\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:04.206358 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.206334    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-e0e300-589877f74c-xnhdg"]
Oct 30 19:46:04.223773 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.223733    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3add95-d9bb45d9f-sjmpj" podUID="c4e830ac-bc83-4389-ac35-f00e5fbdf258" containerName="payload-pre-processing" containerID="cri-o://886f52afe95b72986f54c7d26f121ad65c2a4779643e1b364f0f987a93972222" gracePeriod=25
Oct 30 19:46:04.233479 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.233450    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-e0e300-589877f74c-xnhdg"]
Oct 30 19:46:04.289490 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.289453    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="23069174-9b21-412a-aa6c-118d8446da71" path="/var/lib/kubelet/pods/23069174-9b21-412a-aa6c-118d8446da71/volumes"
Oct 30 19:46:04.290075 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.290055    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="63fc1ff9-fc06-4c20-934f-3667809c844e" path="/var/lib/kubelet/pods/63fc1ff9-fc06-4c20-934f-3667809c844e/volumes"
Oct 30 19:46:04.533832 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.533743    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-3735b2-795c89ffc9-rtdm4"]
Oct 30 19:46:04.543477 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.543447    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-3735b2-795c89ffc9-rtdm4"]
Oct 30 19:46:04.858368 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.858291    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-3735b2-6f754b9b8b-b9rgh"]
Oct 30 19:46:04.902247 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.901895    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-3add95-5f665b5b94-hdzl8"
Oct 30 19:46:04.915686 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.915659    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-3735b2-6f754b9b8b-b9rgh"]
Oct 30 19:46:04.989572 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:04.988807    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3add95-d9bb45d9f-sjmpj"
Oct 30 19:46:05.099662 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.099577    5907 generic.go:356] "Generic (PLEG): container finished" podID="7cd1ce0b-3b85-4701-9f18-abdf8a83e564" containerID="d5b866d9b799e30cde92d90bbacd5c90e0524c681da7f081195e7cbd5c7f85c7" exitCode=0
Oct 30 19:46:05.099662 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.099630    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-3add95-5f665b5b94-hdzl8"
Oct 30 19:46:05.099950 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.099662    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-3add95-5f665b5b94-hdzl8" event={"ID":"7cd1ce0b-3b85-4701-9f18-abdf8a83e564","Type":"ContainerDied","Data":"d5b866d9b799e30cde92d90bbacd5c90e0524c681da7f081195e7cbd5c7f85c7"}
Oct 30 19:46:05.099950 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.099718    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-3add95-5f665b5b94-hdzl8" event={"ID":"7cd1ce0b-3b85-4701-9f18-abdf8a83e564","Type":"ContainerDied","Data":"0a6c56840cc8c322366b8bfedb46dda7ae549c87934781c37df6ed5ff4fb498b"}
Oct 30 19:46:05.099950 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.099754    5907 scope.go:122] "RemoveContainer" containerID="d5b866d9b799e30cde92d90bbacd5c90e0524c681da7f081195e7cbd5c7f85c7"
Oct 30 19:46:05.100229 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.100203    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7cd1ce0b-3b85-4701-9f18-abdf8a83e564-kube-api-access-h46mx\" (UniqueName: \"kubernetes.io/projected/7cd1ce0b-3b85-4701-9f18-abdf8a83e564-kube-api-access-h46mx\") pod \"7cd1ce0b-3b85-4701-9f18-abdf8a83e564\" (UID: \"7cd1ce0b-3b85-4701-9f18-abdf8a83e564\") "
Oct 30 19:46:05.100321 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.100305    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c4e830ac-bc83-4389-ac35-f00e5fbdf258-kube-api-access-d96rq\" (UniqueName: \"kubernetes.io/projected/c4e830ac-bc83-4389-ac35-f00e5fbdf258-kube-api-access-d96rq\") pod \"c4e830ac-bc83-4389-ac35-f00e5fbdf258\" (UID: \"c4e830ac-bc83-4389-ac35-f00e5fbdf258\") "
Oct 30 19:46:05.100463 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.100438    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c4e830ac-bc83-4389-ac35-f00e5fbdf258-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4e830ac-bc83-4389-ac35-f00e5fbdf258-plugins-config-volume\") pod \"c4e830ac-bc83-4389-ac35-f00e5fbdf258\" (UID: \"c4e830ac-bc83-4389-ac35-f00e5fbdf258\") "
Oct 30 19:46:05.100612 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.100489    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7cd1ce0b-3b85-4701-9f18-abdf8a83e564-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7cd1ce0b-3b85-4701-9f18-abdf8a83e564-plugins-config-volume\") pod \"7cd1ce0b-3b85-4701-9f18-abdf8a83e564\" (UID: \"7cd1ce0b-3b85-4701-9f18-abdf8a83e564\") "
Oct 30 19:46:05.100971 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.100941    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4e830ac-bc83-4389-ac35-f00e5fbdf258-plugins-config-volume" pod "c4e830ac-bc83-4389-ac35-f00e5fbdf258" (UID: "c4e830ac-bc83-4389-ac35-f00e5fbdf258"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:05.101092 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.100965    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7cd1ce0b-3b85-4701-9f18-abdf8a83e564-plugins-config-volume" pod "7cd1ce0b-3b85-4701-9f18-abdf8a83e564" (UID: "7cd1ce0b-3b85-4701-9f18-abdf8a83e564"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:05.101166 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.101092    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4e830ac-bc83-4389-ac35-f00e5fbdf258-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:05.101166 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.101117    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7cd1ce0b-3b85-4701-9f18-abdf8a83e564-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:05.103183 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.103158    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4e830ac-bc83-4389-ac35-f00e5fbdf258-kube-api-access-d96rq" pod "c4e830ac-bc83-4389-ac35-f00e5fbdf258" (UID: "c4e830ac-bc83-4389-ac35-f00e5fbdf258"). InnerVolumeSpecName "kube-api-access-d96rq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:05.103317 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.103291    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cd1ce0b-3b85-4701-9f18-abdf8a83e564-kube-api-access-h46mx" pod "7cd1ce0b-3b85-4701-9f18-abdf8a83e564" (UID: "7cd1ce0b-3b85-4701-9f18-abdf8a83e564"). InnerVolumeSpecName "kube-api-access-h46mx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:05.105588 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.105563    5907 generic.go:356] "Generic (PLEG): container finished" podID="c4e830ac-bc83-4389-ac35-f00e5fbdf258" containerID="886f52afe95b72986f54c7d26f121ad65c2a4779643e1b364f0f987a93972222" exitCode=0
Oct 30 19:46:05.105712 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.105607    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3add95-d9bb45d9f-sjmpj" event={"ID":"c4e830ac-bc83-4389-ac35-f00e5fbdf258","Type":"ContainerDied","Data":"886f52afe95b72986f54c7d26f121ad65c2a4779643e1b364f0f987a93972222"}
Oct 30 19:46:05.105712 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.105616    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3add95-d9bb45d9f-sjmpj"
Oct 30 19:46:05.105712 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.105648    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3add95-d9bb45d9f-sjmpj" event={"ID":"c4e830ac-bc83-4389-ac35-f00e5fbdf258","Type":"ContainerDied","Data":"31de82f327bad0278c410c6d7be5a3d0ee775d67474029931a3610037a3e9380"}
Oct 30 19:46:05.141537 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.141501    5907 scope.go:122] "RemoveContainer" containerID="d5b866d9b799e30cde92d90bbacd5c90e0524c681da7f081195e7cbd5c7f85c7"
Oct 30 19:46:05.141894 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:46:05.141852    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5b866d9b799e30cde92d90bbacd5c90e0524c681da7f081195e7cbd5c7f85c7\": container with ID starting with d5b866d9b799e30cde92d90bbacd5c90e0524c681da7f081195e7cbd5c7f85c7 not found: ID does not exist" containerID="d5b866d9b799e30cde92d90bbacd5c90e0524c681da7f081195e7cbd5c7f85c7"
Oct 30 19:46:05.141966 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.141902    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5b866d9b799e30cde92d90bbacd5c90e0524c681da7f081195e7cbd5c7f85c7"} err="failed to get container status \"d5b866d9b799e30cde92d90bbacd5c90e0524c681da7f081195e7cbd5c7f85c7\": rpc error: code = NotFound desc = could not find container \"d5b866d9b799e30cde92d90bbacd5c90e0524c681da7f081195e7cbd5c7f85c7\": container with ID starting with d5b866d9b799e30cde92d90bbacd5c90e0524c681da7f081195e7cbd5c7f85c7 not found: ID does not exist"
Oct 30 19:46:05.141966 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.141923    5907 scope.go:122] "RemoveContainer" containerID="886f52afe95b72986f54c7d26f121ad65c2a4779643e1b364f0f987a93972222"
Oct 30 19:46:05.154455 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.154436    5907 scope.go:122] "RemoveContainer" containerID="886f52afe95b72986f54c7d26f121ad65c2a4779643e1b364f0f987a93972222"
Oct 30 19:46:05.154759 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:46:05.154742    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"886f52afe95b72986f54c7d26f121ad65c2a4779643e1b364f0f987a93972222\": container with ID starting with 886f52afe95b72986f54c7d26f121ad65c2a4779643e1b364f0f987a93972222 not found: ID does not exist" containerID="886f52afe95b72986f54c7d26f121ad65c2a4779643e1b364f0f987a93972222"
Oct 30 19:46:05.154819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.154766    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"886f52afe95b72986f54c7d26f121ad65c2a4779643e1b364f0f987a93972222"} err="failed to get container status \"886f52afe95b72986f54c7d26f121ad65c2a4779643e1b364f0f987a93972222\": rpc error: code = NotFound desc = could not find container \"886f52afe95b72986f54c7d26f121ad65c2a4779643e1b364f0f987a93972222\": container with ID starting with 886f52afe95b72986f54c7d26f121ad65c2a4779643e1b364f0f987a93972222 not found: ID does not exist"
Oct 30 19:46:05.218897 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.207699    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h46mx\" (UniqueName: \"kubernetes.io/projected/7cd1ce0b-3b85-4701-9f18-abdf8a83e564-kube-api-access-h46mx\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:05.219185 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.219160    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d96rq\" (UniqueName: \"kubernetes.io/projected/c4e830ac-bc83-4389-ac35-f00e5fbdf258-kube-api-access-d96rq\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:05.239103 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.239068    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-3add95-d9bb45d9f-sjmpj"]
Oct 30 19:46:05.245498 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.245453    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-3add95-d9bb45d9f-sjmpj"]
Oct 30 19:46:05.331530 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.331499    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7263-9202e9"]
Oct 30 19:46:05.332286 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.332267    5907 cpu_manager.go:414] "containerMap: removing container" podUID="320e693e-a39d-4e8b-9772-cc8e811acc69" containerName="istio-proxy"
Oct 30 19:46:05.332370 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.332287    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="320e693e-a39d-4e8b-9772-cc8e811acc69" containerName="istio-proxy"
Oct 30 19:46:05.332370 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.332304    5907 cpu_manager.go:414] "containerMap: removing container" podUID="97e31b29-cb25-492c-bd66-fcab4b422ab9" containerName="payload-pre-processing"
Oct 30 19:46:05.332370 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.332309    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="97e31b29-cb25-492c-bd66-fcab4b422ab9" containerName="payload-pre-processing"
Oct 30 19:46:05.332370 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.332330    5907 cpu_manager.go:414] "containerMap: removing container" podUID="b1714315-7df0-4860-8759-bdfab7c13824" containerName="payload-processing"
Oct 30 19:46:05.332370 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.332335    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b1714315-7df0-4860-8759-bdfab7c13824" containerName="payload-processing"
Oct 30 19:46:05.332370 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.332359    5907 cpu_manager.go:414] "containerMap: removing container" podUID="7cd1ce0b-3b85-4701-9f18-abdf8a83e564" containerName="payload-processing"
Oct 30 19:46:05.332370 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.332365    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7cd1ce0b-3b85-4701-9f18-abdf8a83e564" containerName="payload-processing"
Oct 30 19:46:05.332612 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.332376    5907 cpu_manager.go:414] "containerMap: removing container" podUID="c4e830ac-bc83-4389-ac35-f00e5fbdf258" containerName="payload-pre-processing"
Oct 30 19:46:05.332612 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.332382    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c4e830ac-bc83-4389-ac35-f00e5fbdf258" containerName="payload-pre-processing"
Oct 30 19:46:05.332612 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.332391    5907 cpu_manager.go:414] "containerMap: removing container" podUID="23069174-9b21-412a-aa6c-118d8446da71" containerName="payload-processing"
Oct 30 19:46:05.332612 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.332395    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="23069174-9b21-412a-aa6c-118d8446da71" containerName="payload-processing"
Oct 30 19:46:05.332612 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.332406    5907 cpu_manager.go:414] "containerMap: removing container" podUID="320e693e-a39d-4e8b-9772-cc8e811acc69" containerName="istio-proxy"
Oct 30 19:46:05.332612 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.332410    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="320e693e-a39d-4e8b-9772-cc8e811acc69" containerName="istio-proxy"
Oct 30 19:46:05.332612 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.332419    5907 cpu_manager.go:414] "containerMap: removing container" podUID="810e8003-26f9-422c-a7e8-9cb10c76ce45" containerName="revoke-keys"
Oct 30 19:46:05.332612 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.332423    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="810e8003-26f9-422c-a7e8-9cb10c76ce45" containerName="revoke-keys"
Oct 30 19:46:05.332612 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.332429    5907 cpu_manager.go:414] "containerMap: removing container" podUID="63fc1ff9-fc06-4c20-934f-3667809c844e" containerName="payload-pre-processing"
Oct 30 19:46:05.332612 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.332433    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="63fc1ff9-fc06-4c20-934f-3667809c844e" containerName="payload-pre-processing"
Oct 30 19:46:05.332612 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.332549    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="7cd1ce0b-3b85-4701-9f18-abdf8a83e564" containerName="payload-processing"
Oct 30 19:46:05.332612 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.332558    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="c4e830ac-bc83-4389-ac35-f00e5fbdf258" containerName="payload-pre-processing"
Oct 30 19:46:05.332612 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.332567    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="23069174-9b21-412a-aa6c-118d8446da71" containerName="payload-processing"
Oct 30 19:46:05.332612 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.332574    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="320e693e-a39d-4e8b-9772-cc8e811acc69" containerName="istio-proxy"
Oct 30 19:46:05.332612 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.332580    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="810e8003-26f9-422c-a7e8-9cb10c76ce45" containerName="revoke-keys"
Oct 30 19:46:05.332612 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.332585    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="63fc1ff9-fc06-4c20-934f-3667809c844e" containerName="payload-pre-processing"
Oct 30 19:46:05.332612 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.332595    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="97e31b29-cb25-492c-bd66-fcab4b422ab9" containerName="payload-pre-processing"
Oct 30 19:46:05.332612 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.332603    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="b1714315-7df0-4860-8759-bdfab7c13824" containerName="payload-processing"
Oct 30 19:46:05.337953 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.337934    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7263-9202e9"
Oct 30 19:46:05.368328 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.368304    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-r79h9\""
Oct 30 19:46:05.372847 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.372824    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7263-9202e9"]
Oct 30 19:46:05.422355 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.422322    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ngdm\" (UniqueName: \"kubernetes.io/projected/26677ae3-486b-4b0a-b640-8e5095b52708-kube-api-access-6ngdm\") pod \"test-curl-7263-9202e9\" (UID: \"26677ae3-486b-4b0a-b640-8e5095b52708\") " pod="opendatahub/test-curl-7263-9202e9"
Oct 30 19:46:05.481035 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.480997    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-3add95-5f665b5b94-hdzl8"]
Oct 30 19:46:05.485009 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.484983    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-3add95-5f665b5b94-hdzl8"]
Oct 30 19:46:05.523242 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.523213    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ngdm\" (UniqueName: \"kubernetes.io/projected/26677ae3-486b-4b0a-b640-8e5095b52708-kube-api-access-6ngdm\") pod \"test-curl-7263-9202e9\" (UID: \"26677ae3-486b-4b0a-b640-8e5095b52708\") " pod="opendatahub/test-curl-7263-9202e9"
Oct 30 19:46:05.545651 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.545619    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ngdm\" (UniqueName: \"kubernetes.io/projected/26677ae3-486b-4b0a-b640-8e5095b52708-kube-api-access-6ngdm\") pod \"test-curl-7263-9202e9\" (UID: \"26677ae3-486b-4b0a-b640-8e5095b52708\") " pod="opendatahub/test-curl-7263-9202e9"
Oct 30 19:46:05.650913 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.650823    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7263-9202e9"
Oct 30 19:46:05.973405 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:46:05.973370    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod26677ae3_486b_4b0a_b640_8e5095b52708.slice/crio-92d36f300a5a181c0b4367650c77e398d14e916dd26df08b64715fc58a676b38 WatchSource:0}: Error finding container 92d36f300a5a181c0b4367650c77e398d14e916dd26df08b64715fc58a676b38: Status 404 returned error can't find the container with id 92d36f300a5a181c0b4367650c77e398d14e916dd26df08b64715fc58a676b38
Oct 30 19:46:05.975133 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:05.975072    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7263-9202e9"]
Oct 30 19:46:06.176607 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:06.176534    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7263-9202e9" event={"ID":"26677ae3-486b-4b0a-b640-8e5095b52708","Type":"ContainerStarted","Data":"92d36f300a5a181c0b4367650c77e398d14e916dd26df08b64715fc58a676b38"}
Oct 30 19:46:06.289343 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:06.289269    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7cd1ce0b-3b85-4701-9f18-abdf8a83e564" path="/var/lib/kubelet/pods/7cd1ce0b-3b85-4701-9f18-abdf8a83e564/volumes"
Oct 30 19:46:06.289737 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:06.289723    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="97e31b29-cb25-492c-bd66-fcab4b422ab9" path="/var/lib/kubelet/pods/97e31b29-cb25-492c-bd66-fcab4b422ab9/volumes"
Oct 30 19:46:06.290111 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:06.290099    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b1714315-7df0-4860-8759-bdfab7c13824" path="/var/lib/kubelet/pods/b1714315-7df0-4860-8759-bdfab7c13824/volumes"
Oct 30 19:46:06.290454 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:06.290443    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c4e830ac-bc83-4389-ac35-f00e5fbdf258" path="/var/lib/kubelet/pods/c4e830ac-bc83-4389-ac35-f00e5fbdf258/volumes"
Oct 30 19:46:07.203566 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:07.203519    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7263-9202e9" event={"ID":"26677ae3-486b-4b0a-b640-8e5095b52708","Type":"ContainerStarted","Data":"aa905cc8fc1b99082f8e6152579de774cfe60114c5fb7efb888941b2dcc4125e"}
Oct 30 19:46:07.275419 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:07.274597    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7263-9202e9" podStartSLOduration=2.274577264 podStartE2EDuration="2.274577264s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:46:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:46:07.269463162 +0000 UTC m=+1479.112395101" watchObservedRunningTime="2026-10-30 19:46:07.274577264 +0000 UTC m=+1479.117509203"
Oct 30 19:46:08.209121 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:08.209086    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7263-9202e9"]
Oct 30 19:46:08.219824 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:08.219787    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7263-9202e9"]
Oct 30 19:46:08.588873 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:08.588780    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-ptkd4"]
Oct 30 19:46:08.589574 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:08.589558    5907 cpu_manager.go:414] "containerMap: removing container" podUID="26677ae3-486b-4b0a-b640-8e5095b52708" containerName="test-curl-7263-9202e9"
Oct 30 19:46:08.589622 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:08.589576    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="26677ae3-486b-4b0a-b640-8e5095b52708" containerName="test-curl-7263-9202e9"
Oct 30 19:46:08.589721 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:08.589704    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="320e693e-a39d-4e8b-9772-cc8e811acc69" containerName="istio-proxy"
Oct 30 19:46:08.589792 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:08.589722    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="26677ae3-486b-4b0a-b640-8e5095b52708" containerName="test-curl-7263-9202e9"
Oct 30 19:46:08.595187 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:08.595166    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-ptkd4"
Oct 30 19:46:08.619001 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:08.618975    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:46:08.630214 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:08.630191    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:46:08.657150 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:08.657121    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-ptkd4"]
Oct 30 19:46:08.697542 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:08.697503    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7e2cfaea-36de-4c52-a106-7b20ffda86e3-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-mt-67953efd-ptkd4\" (UID: \"7e2cfaea-36de-4c52-a106-7b20ffda86e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-ptkd4"
Oct 30 19:46:08.697815 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:08.697707    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w69hs\" (UniqueName: \"kubernetes.io/projected/7e2cfaea-36de-4c52-a106-7b20ffda86e3-kube-api-access-w69hs\") pod \"maas-api-revoke-keys-e2e-mt-67953efd-ptkd4\" (UID: \"7e2cfaea-36de-4c52-a106-7b20ffda86e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-ptkd4"
Oct 30 19:46:08.697815 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:08.697741    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7e2cfaea-36de-4c52-a106-7b20ffda86e3-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-mt-67953efd-ptkd4\" (UID: \"7e2cfaea-36de-4c52-a106-7b20ffda86e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-ptkd4"
Oct 30 19:46:08.798980 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:08.798944    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w69hs\" (UniqueName: \"kubernetes.io/projected/7e2cfaea-36de-4c52-a106-7b20ffda86e3-kube-api-access-w69hs\") pod \"maas-api-revoke-keys-e2e-mt-67953efd-ptkd4\" (UID: \"7e2cfaea-36de-4c52-a106-7b20ffda86e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-ptkd4"
Oct 30 19:46:08.799149 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:08.799009    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7e2cfaea-36de-4c52-a106-7b20ffda86e3-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-mt-67953efd-ptkd4\" (UID: \"7e2cfaea-36de-4c52-a106-7b20ffda86e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-ptkd4"
Oct 30 19:46:08.799214 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:08.799152    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7e2cfaea-36de-4c52-a106-7b20ffda86e3-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-mt-67953efd-ptkd4\" (UID: \"7e2cfaea-36de-4c52-a106-7b20ffda86e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-ptkd4"
Oct 30 19:46:08.800007 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:08.799751    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w6-ef9a5a-59fc6bf976-zfzks" podUID="09920b13-483e-47d3-89f0-c22163568979" containerName="payload-processing" containerID="cri-o://874488b95670c3078fd206e79eb89fb510f58632d62b6a81e4c02f9372f85146" gracePeriod=25
Oct 30 19:46:08.800007 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:08.799790    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7e2cfaea-36de-4c52-a106-7b20ffda86e3-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-mt-67953efd-ptkd4\" (UID: \"7e2cfaea-36de-4c52-a106-7b20ffda86e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-ptkd4"
Oct 30 19:46:08.801802 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:08.801769    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7e2cfaea-36de-4c52-a106-7b20ffda86e3-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-mt-67953efd-ptkd4\" (UID: \"7e2cfaea-36de-4c52-a106-7b20ffda86e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-ptkd4"
Oct 30 19:46:08.821321 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:08.821290    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w69hs\" (UniqueName: \"kubernetes.io/projected/7e2cfaea-36de-4c52-a106-7b20ffda86e3-kube-api-access-w69hs\") pod \"maas-api-revoke-keys-e2e-mt-67953efd-ptkd4\" (UID: \"7e2cfaea-36de-4c52-a106-7b20ffda86e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-ptkd4"
Oct 30 19:46:08.859559 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:08.859439    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ef9a5a-796b5cf9dc-kmcg7" podUID="3b797e1f-bd98-4740-b15d-6ee4e1029d2a" containerName="payload-pre-processing" containerID="cri-o://d3a10aa6120cd43ac32dcff62999d39a94e839e75b0014873b4612e426663b6b" gracePeriod=25
Oct 30 19:46:08.910472 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:08.910438    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-ptkd4"
Oct 30 19:46:09.283293 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:09.283113    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-ptkd4"]
Oct 30 19:46:09.296839 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:09.296698    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7263-9202e9" podUID="26677ae3-486b-4b0a-b640-8e5095b52708" containerName="test-curl-7263-9202e9" containerID="cri-o://aa905cc8fc1b99082f8e6152579de774cfe60114c5fb7efb888941b2dcc4125e" gracePeriod=2
Oct 30 19:46:10.305955 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:10.305922    5907 generic.go:356] "Generic (PLEG): container finished" podID="7e2cfaea-36de-4c52-a106-7b20ffda86e3" containerID="910b58dfdad2ebace5d00060d016f72634c2009a047ead79e973555f8a65e1ec" exitCode=0
Oct 30 19:46:10.306341 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:10.306012    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-ptkd4" event={"ID":"7e2cfaea-36de-4c52-a106-7b20ffda86e3","Type":"ContainerDied","Data":"910b58dfdad2ebace5d00060d016f72634c2009a047ead79e973555f8a65e1ec"}
Oct 30 19:46:10.306341 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:10.306063    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-ptkd4" event={"ID":"7e2cfaea-36de-4c52-a106-7b20ffda86e3","Type":"ContainerStarted","Data":"dcd6d1b7d27e832dd274216aad98a362341c668cf015d336816cd0996f0c87b5"}
Oct 30 19:46:10.996907 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:10.992264    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks"]
Oct 30 19:46:10.996907 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:10.992522    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks" podUID="dde06aaa-256e-46d4-98cf-a762db37475b" containerName="maas-api" containerID="cri-o://46686b1dd6ea0d5d0a901e412801afb72594e73a19c84f2e1c36506e3147ae78" gracePeriod=30
Oct 30 19:46:11.426067 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:11.422649    5907 generic.go:356] "Generic (PLEG): container finished" podID="dde06aaa-256e-46d4-98cf-a762db37475b" containerID="46686b1dd6ea0d5d0a901e412801afb72594e73a19c84f2e1c36506e3147ae78" exitCode=0
Oct 30 19:46:11.426067 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:11.423104    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks" event={"ID":"dde06aaa-256e-46d4-98cf-a762db37475b","Type":"ContainerDied","Data":"46686b1dd6ea0d5d0a901e412801afb72594e73a19c84f2e1c36506e3147ae78"}
Oct 30 19:46:11.485894 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:11.483235    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks"
Oct 30 19:46:11.514895 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:11.512597    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dde06aaa-256e-46d4-98cf-a762db37475b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dde06aaa-256e-46d4-98cf-a762db37475b-maas-api-tls\") pod \"dde06aaa-256e-46d4-98cf-a762db37475b\" (UID: \"dde06aaa-256e-46d4-98cf-a762db37475b\") "
Oct 30 19:46:11.514895 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:11.512646    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dde06aaa-256e-46d4-98cf-a762db37475b-kube-api-access-hz5b6\" (UniqueName: \"kubernetes.io/projected/dde06aaa-256e-46d4-98cf-a762db37475b-kube-api-access-hz5b6\") pod \"dde06aaa-256e-46d4-98cf-a762db37475b\" (UID: \"dde06aaa-256e-46d4-98cf-a762db37475b\") "
Oct 30 19:46:11.514895 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:11.512819    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dde06aaa-256e-46d4-98cf-a762db37475b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dde06aaa-256e-46d4-98cf-a762db37475b-openshift-service-ca\") pod \"dde06aaa-256e-46d4-98cf-a762db37475b\" (UID: \"dde06aaa-256e-46d4-98cf-a762db37475b\") "
Oct 30 19:46:11.514895 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:11.512967    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dde06aaa-256e-46d4-98cf-a762db37475b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/dde06aaa-256e-46d4-98cf-a762db37475b-metrics-certs\") pod \"dde06aaa-256e-46d4-98cf-a762db37475b\" (UID: \"dde06aaa-256e-46d4-98cf-a762db37475b\") "
Oct 30 19:46:11.528890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:11.526947    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dde06aaa-256e-46d4-98cf-a762db37475b-openshift-service-ca" pod "dde06aaa-256e-46d4-98cf-a762db37475b" (UID: "dde06aaa-256e-46d4-98cf-a762db37475b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:11.697765 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:11.574473    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dde06aaa-256e-46d4-98cf-a762db37475b-kube-api-access-hz5b6" pod "dde06aaa-256e-46d4-98cf-a762db37475b" (UID: "dde06aaa-256e-46d4-98cf-a762db37475b"). InnerVolumeSpecName "kube-api-access-hz5b6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:11.697765 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:11.574681    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dde06aaa-256e-46d4-98cf-a762db37475b-metrics-certs" pod "dde06aaa-256e-46d4-98cf-a762db37475b" (UID: "dde06aaa-256e-46d4-98cf-a762db37475b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:46:11.697765 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:11.642508    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dde06aaa-256e-46d4-98cf-a762db37475b-maas-api-tls" pod "dde06aaa-256e-46d4-98cf-a762db37475b" (UID: "dde06aaa-256e-46d4-98cf-a762db37475b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:46:11.697765 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:11.665184    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dde06aaa-256e-46d4-98cf-a762db37475b-maas-api-tls\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:11.697765 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:11.665230    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hz5b6\" (UniqueName: \"kubernetes.io/projected/dde06aaa-256e-46d4-98cf-a762db37475b-kube-api-access-hz5b6\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:11.697765 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:11.665250    5907 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dde06aaa-256e-46d4-98cf-a762db37475b-openshift-service-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:11.697765 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:11.665265    5907 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dde06aaa-256e-46d4-98cf-a762db37475b-metrics-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:11.905224 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:11.905191    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-ptkd4"
Oct 30 19:46:11.942092 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:11.942068    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7263-9202e9"
Oct 30 19:46:11.971415 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:11.971306    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7e2cfaea-36de-4c52-a106-7b20ffda86e3-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7e2cfaea-36de-4c52-a106-7b20ffda86e3-maas-api-openshift-ca\") pod \"7e2cfaea-36de-4c52-a106-7b20ffda86e3\" (UID: \"7e2cfaea-36de-4c52-a106-7b20ffda86e3\") "
Oct 30 19:46:11.971592 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:11.971434    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7e2cfaea-36de-4c52-a106-7b20ffda86e3-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7e2cfaea-36de-4c52-a106-7b20ffda86e3-maas-api-xks-ca\") pod \"7e2cfaea-36de-4c52-a106-7b20ffda86e3\" (UID: \"7e2cfaea-36de-4c52-a106-7b20ffda86e3\") "
Oct 30 19:46:11.971592 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:11.971489    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/26677ae3-486b-4b0a-b640-8e5095b52708-kube-api-access-6ngdm\" (UniqueName: \"kubernetes.io/projected/26677ae3-486b-4b0a-b640-8e5095b52708-kube-api-access-6ngdm\") pod \"26677ae3-486b-4b0a-b640-8e5095b52708\" (UID: \"26677ae3-486b-4b0a-b640-8e5095b52708\") "
Oct 30 19:46:11.971716 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:11.971675    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7e2cfaea-36de-4c52-a106-7b20ffda86e3-kube-api-access-w69hs\" (UniqueName: \"kubernetes.io/projected/7e2cfaea-36de-4c52-a106-7b20ffda86e3-kube-api-access-w69hs\") pod \"7e2cfaea-36de-4c52-a106-7b20ffda86e3\" (UID: \"7e2cfaea-36de-4c52-a106-7b20ffda86e3\") "
Oct 30 19:46:11.971780 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:11.971721    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e2cfaea-36de-4c52-a106-7b20ffda86e3-maas-api-openshift-ca" pod "7e2cfaea-36de-4c52-a106-7b20ffda86e3" (UID: "7e2cfaea-36de-4c52-a106-7b20ffda86e3"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:11.972415 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:11.972386    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7e2cfaea-36de-4c52-a106-7b20ffda86e3-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:11.975205 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:11.975182    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e2cfaea-36de-4c52-a106-7b20ffda86e3-kube-api-access-w69hs" pod "7e2cfaea-36de-4c52-a106-7b20ffda86e3" (UID: "7e2cfaea-36de-4c52-a106-7b20ffda86e3"). InnerVolumeSpecName "kube-api-access-w69hs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:11.975982 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:11.975953    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e2cfaea-36de-4c52-a106-7b20ffda86e3-maas-api-xks-ca" pod "7e2cfaea-36de-4c52-a106-7b20ffda86e3" (UID: "7e2cfaea-36de-4c52-a106-7b20ffda86e3"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:46:11.977892 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:11.977855    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26677ae3-486b-4b0a-b640-8e5095b52708-kube-api-access-6ngdm" pod "26677ae3-486b-4b0a-b640-8e5095b52708" (UID: "26677ae3-486b-4b0a-b640-8e5095b52708"). InnerVolumeSpecName "kube-api-access-6ngdm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:12.081884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:12.075735    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w69hs\" (UniqueName: \"kubernetes.io/projected/7e2cfaea-36de-4c52-a106-7b20ffda86e3-kube-api-access-w69hs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:12.081884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:12.075777    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7e2cfaea-36de-4c52-a106-7b20ffda86e3-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:12.081884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:12.075802    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ngdm\" (UniqueName: \"kubernetes.io/projected/26677ae3-486b-4b0a-b640-8e5095b52708-kube-api-access-6ngdm\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:12.291648 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:12.291568    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="26677ae3-486b-4b0a-b640-8e5095b52708" path="/var/lib/kubelet/pods/26677ae3-486b-4b0a-b640-8e5095b52708/volumes"
Oct 30 19:46:12.432827 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:12.432791    5907 generic.go:356] "Generic (PLEG): container finished" podID="26677ae3-486b-4b0a-b640-8e5095b52708" containerID="aa905cc8fc1b99082f8e6152579de774cfe60114c5fb7efb888941b2dcc4125e" exitCode=137
Oct 30 19:46:12.433298 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:12.432955    5907 scope.go:122] "RemoveContainer" containerID="aa905cc8fc1b99082f8e6152579de774cfe60114c5fb7efb888941b2dcc4125e"
Oct 30 19:46:12.433298 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:12.432966    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7263-9202e9"
Oct 30 19:46:12.435820 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:12.435110    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks"
Oct 30 19:46:12.435820 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:12.435188    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks" event={"ID":"dde06aaa-256e-46d4-98cf-a762db37475b","Type":"ContainerDied","Data":"73906159cc3aed1874ee5b5e5eddc822012f2095d273f5181eecce856df2bf80"}
Oct 30 19:46:12.436943 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:12.436924    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-ptkd4"
Oct 30 19:46:12.437055 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:12.436966    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-ptkd4" event={"ID":"7e2cfaea-36de-4c52-a106-7b20ffda86e3","Type":"ContainerDied","Data":"dcd6d1b7d27e832dd274216aad98a362341c668cf015d336816cd0996f0c87b5"}
Oct 30 19:46:12.437055 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:12.437002    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dcd6d1b7d27e832dd274216aad98a362341c668cf015d336816cd0996f0c87b5"
Oct 30 19:46:12.446812 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:12.446798    5907 scope.go:122] "RemoveContainer" containerID="aa905cc8fc1b99082f8e6152579de774cfe60114c5fb7efb888941b2dcc4125e"
Oct 30 19:46:12.447101 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:46:12.447083    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa905cc8fc1b99082f8e6152579de774cfe60114c5fb7efb888941b2dcc4125e\": container with ID starting with aa905cc8fc1b99082f8e6152579de774cfe60114c5fb7efb888941b2dcc4125e not found: ID does not exist" containerID="aa905cc8fc1b99082f8e6152579de774cfe60114c5fb7efb888941b2dcc4125e"
Oct 30 19:46:12.447201 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:12.447111    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa905cc8fc1b99082f8e6152579de774cfe60114c5fb7efb888941b2dcc4125e"} err="failed to get container status \"aa905cc8fc1b99082f8e6152579de774cfe60114c5fb7efb888941b2dcc4125e\": rpc error: code = NotFound desc = could not find container \"aa905cc8fc1b99082f8e6152579de774cfe60114c5fb7efb888941b2dcc4125e\": container with ID starting with aa905cc8fc1b99082f8e6152579de774cfe60114c5fb7efb888941b2dcc4125e not found: ID does not exist"
Oct 30 19:46:12.447201 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:12.447135    5907 scope.go:122] "RemoveContainer" containerID="46686b1dd6ea0d5d0a901e412801afb72594e73a19c84f2e1c36506e3147ae78"
Oct 30 19:46:12.500432 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:12.500396    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks"]
Oct 30 19:46:12.513759 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:12.513734    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-e0e300-99b88d8b8-njbks"]
Oct 30 19:46:13.101979 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:13.101945    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" containerID="cri-o://a8b2360b8d75055e61f5ff88e70e47605769fc66711280e2b5e3b4f7b146dd4a" gracePeriod=45
Oct 30 19:46:13.105183 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:13.105129    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.103:8000/ready\": read tcp 10.134.0.2:36794->10.134.0.103:8000: read: connection reset by peer"
Oct 30 19:46:13.380310 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:13.380283    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:46:13.395373 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:13.395344    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-tmp-dir\") pod \"cf66ca70-f9e4-4248-9ae7-ee00390ce05a\" (UID: \"cf66ca70-f9e4-4248-9ae7-ee00390ce05a\") "
Oct 30 19:46:13.395536 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:13.395435    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-tls-certs\" (UniqueName: \"kubernetes.io/secret/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-tls-certs\") pod \"cf66ca70-f9e4-4248-9ae7-ee00390ce05a\" (UID: \"cf66ca70-f9e4-4248-9ae7-ee00390ce05a\") "
Oct 30 19:46:13.395536 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:13.395485    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-home\" (UniqueName: \"kubernetes.io/empty-dir/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-home\") pod \"cf66ca70-f9e4-4248-9ae7-ee00390ce05a\" (UID: \"cf66ca70-f9e4-4248-9ae7-ee00390ce05a\") "
Oct 30 19:46:13.395675 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:13.395654    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-tmp-dir" pod "cf66ca70-f9e4-4248-9ae7-ee00390ce05a" (UID: "cf66ca70-f9e4-4248-9ae7-ee00390ce05a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:13.395734 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:13.395681    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-model-cache\") pod \"cf66ca70-f9e4-4248-9ae7-ee00390ce05a\" (UID: \"cf66ca70-f9e4-4248-9ae7-ee00390ce05a\") "
Oct 30 19:46:13.395791 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:13.395753    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-home" pod "cf66ca70-f9e4-4248-9ae7-ee00390ce05a" (UID: "cf66ca70-f9e4-4248-9ae7-ee00390ce05a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:13.395850 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:13.395793    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-kube-api-access-cklh9\" (UniqueName: \"kubernetes.io/projected/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-kube-api-access-cklh9\") pod \"cf66ca70-f9e4-4248-9ae7-ee00390ce05a\" (UID: \"cf66ca70-f9e4-4248-9ae7-ee00390ce05a\") "
Oct 30 19:46:13.395934 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:13.395915    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-model-cache" pod "cf66ca70-f9e4-4248-9ae7-ee00390ce05a" (UID: "cf66ca70-f9e4-4248-9ae7-ee00390ce05a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:13.395991 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:13.395954    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-dshm\") pod \"cf66ca70-f9e4-4248-9ae7-ee00390ce05a\" (UID: \"cf66ca70-f9e4-4248-9ae7-ee00390ce05a\") "
Oct 30 19:46:13.396738 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:13.396710    5907 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-home\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:13.396738 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:13.396739    5907 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-model-cache\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:13.396985 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:13.396754    5907 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-tmp-dir\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:13.398210 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:13.398184    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-tls-certs" pod "cf66ca70-f9e4-4248-9ae7-ee00390ce05a" (UID: "cf66ca70-f9e4-4248-9ae7-ee00390ce05a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:46:13.398210 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:13.398197    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-kube-api-access-cklh9" pod "cf66ca70-f9e4-4248-9ae7-ee00390ce05a" (UID: "cf66ca70-f9e4-4248-9ae7-ee00390ce05a"). InnerVolumeSpecName "kube-api-access-cklh9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:13.398417 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:13.398402    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-dshm" pod "cf66ca70-f9e4-4248-9ae7-ee00390ce05a" (UID: "cf66ca70-f9e4-4248-9ae7-ee00390ce05a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:13.453917 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:13.452891    5907 generic.go:356] "Generic (PLEG): container finished" podID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerID="a8b2360b8d75055e61f5ff88e70e47605769fc66711280e2b5e3b4f7b146dd4a" exitCode=0
Oct 30 19:46:13.453917 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:13.452958    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" event={"ID":"cf66ca70-f9e4-4248-9ae7-ee00390ce05a","Type":"ContainerDied","Data":"a8b2360b8d75055e61f5ff88e70e47605769fc66711280e2b5e3b4f7b146dd4a"}
Oct 30 19:46:13.453917 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:13.453022    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx" event={"ID":"cf66ca70-f9e4-4248-9ae7-ee00390ce05a","Type":"ContainerDied","Data":"52b3ebb7aa1cdb227486a19c314c85401a6e53999472ded555faac8b9ace16e8"}
Oct 30 19:46:13.453917 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:13.452965    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"
Oct 30 19:46:13.453917 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:13.453048    5907 scope.go:122] "RemoveContainer" containerID="a8b2360b8d75055e61f5ff88e70e47605769fc66711280e2b5e3b4f7b146dd4a"
Oct 30 19:46:13.474666 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:13.474641    5907 scope.go:122] "RemoveContainer" containerID="a8b2360b8d75055e61f5ff88e70e47605769fc66711280e2b5e3b4f7b146dd4a"
Oct 30 19:46:13.475184 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:46:13.475158    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8b2360b8d75055e61f5ff88e70e47605769fc66711280e2b5e3b4f7b146dd4a\": container with ID starting with a8b2360b8d75055e61f5ff88e70e47605769fc66711280e2b5e3b4f7b146dd4a not found: ID does not exist" containerID="a8b2360b8d75055e61f5ff88e70e47605769fc66711280e2b5e3b4f7b146dd4a"
Oct 30 19:46:13.475281 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:13.475191    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8b2360b8d75055e61f5ff88e70e47605769fc66711280e2b5e3b4f7b146dd4a"} err="failed to get container status \"a8b2360b8d75055e61f5ff88e70e47605769fc66711280e2b5e3b4f7b146dd4a\": rpc error: code = NotFound desc = could not find container \"a8b2360b8d75055e61f5ff88e70e47605769fc66711280e2b5e3b4f7b146dd4a\": container with ID starting with a8b2360b8d75055e61f5ff88e70e47605769fc66711280e2b5e3b4f7b146dd4a not found: ID does not exist"
Oct 30 19:46:13.497747 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:13.497714    5907 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-tls-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:13.497747 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:13.497746    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cklh9\" (UniqueName: \"kubernetes.io/projected/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-kube-api-access-cklh9\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:13.497959 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:13.497762    5907 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf66ca70-f9e4-4248-9ae7-ee00390ce05a-dshm\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:13.519181 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:13.519155    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"]
Oct 30 19:46:13.524676 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:13.524650    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-ef9a5a/distinct-2-w6-ef9a5a-kserve-9cbcd9c87-lqkxx"]
Oct 30 19:46:14.005568 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:14.005503    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ef9a5a-796b5cf9dc-kmcg7" podUID="3b797e1f-bd98-4740-b15d-6ee4e1029d2a" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:46:14.005774 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:14.005502    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w6-ef9a5a-59fc6bf976-zfzks" podUID="09920b13-483e-47d3-89f0-c22163568979" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:46:14.289620 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:14.289527    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" path="/var/lib/kubelet/pods/cf66ca70-f9e4-4248-9ae7-ee00390ce05a/volumes"
Oct 30 19:46:14.290089 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:14.290072    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dde06aaa-256e-46d4-98cf-a762db37475b" path="/var/lib/kubelet/pods/dde06aaa-256e-46d4-98cf-a762db37475b/volumes"
Oct 30 19:46:17.505452 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:46:17.505410    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:46:18.986263 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:18.986229    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-67953efd-b8797cccd-pr7bp"]
Oct 30 19:46:19.020302 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:19.020268    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-mhv64"]
Oct 30 19:46:20.589487 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.589450    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-57e671e1-fjvwq"]
Oct 30 19:46:20.590541 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.590519    5907 cpu_manager.go:414] "containerMap: removing container" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main"
Oct 30 19:46:20.590656 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.590541    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main"
Oct 30 19:46:20.590656 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.590572    5907 cpu_manager.go:414] "containerMap: removing container" podUID="dde06aaa-256e-46d4-98cf-a762db37475b" containerName="maas-api"
Oct 30 19:46:20.590656 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.590580    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dde06aaa-256e-46d4-98cf-a762db37475b" containerName="maas-api"
Oct 30 19:46:20.590656 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.590633    5907 cpu_manager.go:414] "containerMap: removing container" podUID="7e2cfaea-36de-4c52-a106-7b20ffda86e3" containerName="revoke-keys"
Oct 30 19:46:20.590656 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.590641    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7e2cfaea-36de-4c52-a106-7b20ffda86e3" containerName="revoke-keys"
Oct 30 19:46:20.590835 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.590773    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf66ca70-f9e4-4248-9ae7-ee00390ce05a" containerName="main"
Oct 30 19:46:20.590835 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.590788    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="dde06aaa-256e-46d4-98cf-a762db37475b" containerName="maas-api"
Oct 30 19:46:20.590835 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.590805    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="7e2cfaea-36de-4c52-a106-7b20ffda86e3" containerName="revoke-keys"
Oct 30 19:46:20.595255 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.595237    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-57e671e1-fjvwq"
Oct 30 19:46:20.619539 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.619512    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-57e671e1-fjvwq"]
Oct 30 19:46:20.627566 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.627538    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9dd73dbe-af1c-4d56-82e3-228644efda7a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-57e671e1-fjvwq\" (UID: \"9dd73dbe-af1c-4d56-82e3-228644efda7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-57e671e1-fjvwq"
Oct 30 19:46:20.627765 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.627742    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvcc2\" (UniqueName: \"kubernetes.io/projected/9dd73dbe-af1c-4d56-82e3-228644efda7a-kube-api-access-dvcc2\") pod \"maas-api-revoke-sub-57e671e1-fjvwq\" (UID: \"9dd73dbe-af1c-4d56-82e3-228644efda7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-57e671e1-fjvwq"
Oct 30 19:46:20.628251 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.628225    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:46:20.628364 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.628305    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:46:20.628364 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.628341    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9dd73dbe-af1c-4d56-82e3-228644efda7a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-57e671e1-fjvwq\" (UID: \"9dd73dbe-af1c-4d56-82e3-228644efda7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-57e671e1-fjvwq"
Oct 30 19:46:20.641163 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.636373    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7a47e70e-xrzvf"]
Oct 30 19:46:20.642900 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.642199    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a47e70e-xrzvf"
Oct 30 19:46:20.663841 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.658376    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-67953efd-956bfdb9-4ftfp"]
Oct 30 19:46:20.663841 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.658571    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67953efd-956bfdb9-4ftfp" podUID="364ccf73-2cd0-4922-9460-daca3b6f44e1" containerName="maas-api" containerID="cri-o://f4bb8b956dfa105291eaef426473484403ff188cb259a28d564513c3619069c0" gracePeriod=30
Oct 30 19:46:20.664912 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.664662    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7a47e70e-xrzvf"]
Oct 30 19:46:20.729277 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.729248    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvcc2\" (UniqueName: \"kubernetes.io/projected/9dd73dbe-af1c-4d56-82e3-228644efda7a-kube-api-access-dvcc2\") pod \"maas-api-revoke-sub-57e671e1-fjvwq\" (UID: \"9dd73dbe-af1c-4d56-82e3-228644efda7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-57e671e1-fjvwq"
Oct 30 19:46:20.729419 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.729285    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7t7r\" (UniqueName: \"kubernetes.io/projected/b9d6296e-59ef-4853-8470-d851752d8694-kube-api-access-p7t7r\") pod \"maas-api-revoke-sub-7a47e70e-xrzvf\" (UID: \"b9d6296e-59ef-4853-8470-d851752d8694\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a47e70e-xrzvf"
Oct 30 19:46:20.729419 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.729356    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9dd73dbe-af1c-4d56-82e3-228644efda7a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-57e671e1-fjvwq\" (UID: \"9dd73dbe-af1c-4d56-82e3-228644efda7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-57e671e1-fjvwq"
Oct 30 19:46:20.729419 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.729395    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b9d6296e-59ef-4853-8470-d851752d8694-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7a47e70e-xrzvf\" (UID: \"b9d6296e-59ef-4853-8470-d851752d8694\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a47e70e-xrzvf"
Oct 30 19:46:20.729690 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.729640    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9dd73dbe-af1c-4d56-82e3-228644efda7a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-57e671e1-fjvwq\" (UID: \"9dd73dbe-af1c-4d56-82e3-228644efda7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-57e671e1-fjvwq"
Oct 30 19:46:20.729765 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.729694    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b9d6296e-59ef-4853-8470-d851752d8694-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7a47e70e-xrzvf\" (UID: \"b9d6296e-59ef-4853-8470-d851752d8694\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a47e70e-xrzvf"
Oct 30 19:46:20.730295 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.730270    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9dd73dbe-af1c-4d56-82e3-228644efda7a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-57e671e1-fjvwq\" (UID: \"9dd73dbe-af1c-4d56-82e3-228644efda7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-57e671e1-fjvwq"
Oct 30 19:46:20.732015 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.731995    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9dd73dbe-af1c-4d56-82e3-228644efda7a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-57e671e1-fjvwq\" (UID: \"9dd73dbe-af1c-4d56-82e3-228644efda7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-57e671e1-fjvwq"
Oct 30 19:46:20.757104 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.757082    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvcc2\" (UniqueName: \"kubernetes.io/projected/9dd73dbe-af1c-4d56-82e3-228644efda7a-kube-api-access-dvcc2\") pod \"maas-api-revoke-sub-57e671e1-fjvwq\" (UID: \"9dd73dbe-af1c-4d56-82e3-228644efda7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-57e671e1-fjvwq"
Oct 30 19:46:20.830916 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.830881    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7t7r\" (UniqueName: \"kubernetes.io/projected/b9d6296e-59ef-4853-8470-d851752d8694-kube-api-access-p7t7r\") pod \"maas-api-revoke-sub-7a47e70e-xrzvf\" (UID: \"b9d6296e-59ef-4853-8470-d851752d8694\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a47e70e-xrzvf"
Oct 30 19:46:20.831090 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.830999    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b9d6296e-59ef-4853-8470-d851752d8694-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7a47e70e-xrzvf\" (UID: \"b9d6296e-59ef-4853-8470-d851752d8694\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a47e70e-xrzvf"
Oct 30 19:46:20.831158 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.831107    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b9d6296e-59ef-4853-8470-d851752d8694-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7a47e70e-xrzvf\" (UID: \"b9d6296e-59ef-4853-8470-d851752d8694\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a47e70e-xrzvf"
Oct 30 19:46:20.831748 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.831723    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b9d6296e-59ef-4853-8470-d851752d8694-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7a47e70e-xrzvf\" (UID: \"b9d6296e-59ef-4853-8470-d851752d8694\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a47e70e-xrzvf"
Oct 30 19:46:20.833441 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.833419    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b9d6296e-59ef-4853-8470-d851752d8694-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7a47e70e-xrzvf\" (UID: \"b9d6296e-59ef-4853-8470-d851752d8694\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a47e70e-xrzvf"
Oct 30 19:46:20.859777 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.859720    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7t7r\" (UniqueName: \"kubernetes.io/projected/b9d6296e-59ef-4853-8470-d851752d8694-kube-api-access-p7t7r\") pod \"maas-api-revoke-sub-7a47e70e-xrzvf\" (UID: \"b9d6296e-59ef-4853-8470-d851752d8694\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a47e70e-xrzvf"
Oct 30 19:46:20.918935 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:20.912690    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-57e671e1-fjvwq"
Oct 30 19:46:21.049931 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.011579    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a47e70e-xrzvf"
Oct 30 19:46:21.151270 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.151240    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-3add95-864947fbf5-59nqp"]
Oct 30 19:46:21.151496 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.151471    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3add95-864947fbf5-59nqp" podUID="b1523289-eee0-4b86-a62f-ee110dde58e6" containerName="maas-api" containerID="cri-o://0b76b148be93900aee4a78ab49ba7f5ea5bf61d4fcc1c2526e0415ea81d9b3d2" gracePeriod=30
Oct 30 19:46:21.174955 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.174931    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67953efd-956bfdb9-4ftfp"
Oct 30 19:46:21.251482 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.251450    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/364ccf73-2cd0-4922-9460-daca3b6f44e1-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/364ccf73-2cd0-4922-9460-daca3b6f44e1-maas-api-tls\") pod \"364ccf73-2cd0-4922-9460-daca3b6f44e1\" (UID: \"364ccf73-2cd0-4922-9460-daca3b6f44e1\") "
Oct 30 19:46:21.251626 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.251514    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/364ccf73-2cd0-4922-9460-daca3b6f44e1-kube-api-access-tsb59\" (UniqueName: \"kubernetes.io/projected/364ccf73-2cd0-4922-9460-daca3b6f44e1-kube-api-access-tsb59\") pod \"364ccf73-2cd0-4922-9460-daca3b6f44e1\" (UID: \"364ccf73-2cd0-4922-9460-daca3b6f44e1\") "
Oct 30 19:46:21.251626 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.251547    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/364ccf73-2cd0-4922-9460-daca3b6f44e1-metrics-certs\" (UniqueName: \"kubernetes.io/secret/364ccf73-2cd0-4922-9460-daca3b6f44e1-metrics-certs\") pod \"364ccf73-2cd0-4922-9460-daca3b6f44e1\" (UID: \"364ccf73-2cd0-4922-9460-daca3b6f44e1\") "
Oct 30 19:46:21.251824 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.251628    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/364ccf73-2cd0-4922-9460-daca3b6f44e1-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/364ccf73-2cd0-4922-9460-daca3b6f44e1-openshift-service-ca\") pod \"364ccf73-2cd0-4922-9460-daca3b6f44e1\" (UID: \"364ccf73-2cd0-4922-9460-daca3b6f44e1\") "
Oct 30 19:46:21.253424 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.253373    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/364ccf73-2cd0-4922-9460-daca3b6f44e1-openshift-service-ca" pod "364ccf73-2cd0-4922-9460-daca3b6f44e1" (UID: "364ccf73-2cd0-4922-9460-daca3b6f44e1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:21.255132 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.255071    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/364ccf73-2cd0-4922-9460-daca3b6f44e1-kube-api-access-tsb59" pod "364ccf73-2cd0-4922-9460-daca3b6f44e1" (UID: "364ccf73-2cd0-4922-9460-daca3b6f44e1"). InnerVolumeSpecName "kube-api-access-tsb59". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:21.257732 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.257695    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/364ccf73-2cd0-4922-9460-daca3b6f44e1-maas-api-tls" pod "364ccf73-2cd0-4922-9460-daca3b6f44e1" (UID: "364ccf73-2cd0-4922-9460-daca3b6f44e1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:46:21.260640 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.260601    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/364ccf73-2cd0-4922-9460-daca3b6f44e1-metrics-certs" pod "364ccf73-2cd0-4922-9460-daca3b6f44e1" (UID: "364ccf73-2cd0-4922-9460-daca3b6f44e1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:46:21.320420 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.319769    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"]
Oct 30 19:46:21.320420 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.320059    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv" podUID="57838872-c482-41ec-bda0-4a517fb81dcd" containerName="istio-proxy" containerID="cri-o://7d5cbf1113d7cdb51c3e45581ca40732e044dbc3b8d173681cf18aedc4290c97" gracePeriod=30
Oct 30 19:46:21.355189 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.355151    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/364ccf73-2cd0-4922-9460-daca3b6f44e1-maas-api-tls\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:21.355418 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.355400    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tsb59\" (UniqueName: \"kubernetes.io/projected/364ccf73-2cd0-4922-9460-daca3b6f44e1-kube-api-access-tsb59\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:21.355582 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.355565    5907 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/364ccf73-2cd0-4922-9460-daca3b6f44e1-metrics-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:21.355712 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.355699    5907 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/364ccf73-2cd0-4922-9460-daca3b6f44e1-openshift-service-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:21.445337 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.445296    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"]
Oct 30 19:46:21.445636 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.445589    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc" podUID="282f4fd8-512c-41b9-9989-0b84f2de1ce5" containerName="istio-proxy" containerID="cri-o://269422201e7a715b94c4696784ba9bc6fc8a305ce1ef1a2aef55601d94590ecf" gracePeriod=30
Oct 30 19:46:21.459139 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.459102    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"]
Oct 30 19:46:21.459477 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.459354    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c" podUID="81eea74f-1fd9-41b0-8ecf-0501402c6a66" containerName="istio-proxy" containerID="cri-o://5586215f406d71926acf24937a198f86cc4e4be5e19c8e9e111c314a4823a94d" gracePeriod=30
Oct 30 19:46:21.489148 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.489069    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3add95-864947fbf5-59nqp"
Oct 30 19:46:21.489551 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:46:21.489520    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9d6296e_59ef_4853_8470_d851752d8694.slice/crio-6f25b3f2a29e45c5c222a9d9fd15b24f2e20ce031690d428b73c780d63904c3e WatchSource:0}: Error finding container 6f25b3f2a29e45c5c222a9d9fd15b24f2e20ce031690d428b73c780d63904c3e: Status 404 returned error can't find the container with id 6f25b3f2a29e45c5c222a9d9fd15b24f2e20ce031690d428b73c780d63904c3e
Oct 30 19:46:21.490162 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.490137    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7a47e70e-xrzvf"]
Oct 30 19:46:21.518615 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.518584    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-57e671e1-fjvwq"]
Oct 30 19:46:21.518822 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:46:21.518639    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dd73dbe_af1c_4d56_82e3_228644efda7a.slice/crio-cc26acf30dedcff8095c7ecc338aac39b3e35ed0d2fb5ebb6c5325e3fa57fe2b WatchSource:0}: Error finding container cc26acf30dedcff8095c7ecc338aac39b3e35ed0d2fb5ebb6c5325e3fa57fe2b: Status 404 returned error can't find the container with id cc26acf30dedcff8095c7ecc338aac39b3e35ed0d2fb5ebb6c5325e3fa57fe2b
Oct 30 19:46:21.527719 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.527691    5907 generic.go:356] "Generic (PLEG): container finished" podID="b1523289-eee0-4b86-a62f-ee110dde58e6" containerID="0b76b148be93900aee4a78ab49ba7f5ea5bf61d4fcc1c2526e0415ea81d9b3d2" exitCode=0
Oct 30 19:46:21.527823 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.527773    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3add95-864947fbf5-59nqp" event={"ID":"b1523289-eee0-4b86-a62f-ee110dde58e6","Type":"ContainerDied","Data":"0b76b148be93900aee4a78ab49ba7f5ea5bf61d4fcc1c2526e0415ea81d9b3d2"}
Oct 30 19:46:21.527904 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.527820    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3add95-864947fbf5-59nqp" event={"ID":"b1523289-eee0-4b86-a62f-ee110dde58e6","Type":"ContainerDied","Data":"077e5a61ae8776865614dde5ca3d7dcefe0e816a0af92280992a09b31aabdeb0"}
Oct 30 19:46:21.527904 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.527853    5907 scope.go:122] "RemoveContainer" containerID="0b76b148be93900aee4a78ab49ba7f5ea5bf61d4fcc1c2526e0415ea81d9b3d2"
Oct 30 19:46:21.528018 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.528010    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3add95-864947fbf5-59nqp"
Oct 30 19:46:21.530168 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.530142    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a47e70e-xrzvf" event={"ID":"b9d6296e-59ef-4853-8470-d851752d8694","Type":"ContainerStarted","Data":"6f25b3f2a29e45c5c222a9d9fd15b24f2e20ce031690d428b73c780d63904c3e"}
Oct 30 19:46:21.531927 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.531895    5907 generic.go:356] "Generic (PLEG): container finished" podID="364ccf73-2cd0-4922-9460-daca3b6f44e1" containerID="f4bb8b956dfa105291eaef426473484403ff188cb259a28d564513c3619069c0" exitCode=0
Oct 30 19:46:21.532035 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.531938    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67953efd-956bfdb9-4ftfp" event={"ID":"364ccf73-2cd0-4922-9460-daca3b6f44e1","Type":"ContainerDied","Data":"f4bb8b956dfa105291eaef426473484403ff188cb259a28d564513c3619069c0"}
Oct 30 19:46:21.532095 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.532056    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67953efd-956bfdb9-4ftfp" event={"ID":"364ccf73-2cd0-4922-9460-daca3b6f44e1","Type":"ContainerDied","Data":"d959330f652b2831644d84a95abcd2820c381a6c85a924efc2f5843d656570b0"}
Oct 30 19:46:21.532176 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.532122    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67953efd-956bfdb9-4ftfp"
Oct 30 19:46:21.550024 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.550003    5907 scope.go:122] "RemoveContainer" containerID="0b76b148be93900aee4a78ab49ba7f5ea5bf61d4fcc1c2526e0415ea81d9b3d2"
Oct 30 19:46:21.550312 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:46:21.550293    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b76b148be93900aee4a78ab49ba7f5ea5bf61d4fcc1c2526e0415ea81d9b3d2\": container with ID starting with 0b76b148be93900aee4a78ab49ba7f5ea5bf61d4fcc1c2526e0415ea81d9b3d2 not found: ID does not exist" containerID="0b76b148be93900aee4a78ab49ba7f5ea5bf61d4fcc1c2526e0415ea81d9b3d2"
Oct 30 19:46:21.550359 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.550320    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b76b148be93900aee4a78ab49ba7f5ea5bf61d4fcc1c2526e0415ea81d9b3d2"} err="failed to get container status \"0b76b148be93900aee4a78ab49ba7f5ea5bf61d4fcc1c2526e0415ea81d9b3d2\": rpc error: code = NotFound desc = could not find container \"0b76b148be93900aee4a78ab49ba7f5ea5bf61d4fcc1c2526e0415ea81d9b3d2\": container with ID starting with 0b76b148be93900aee4a78ab49ba7f5ea5bf61d4fcc1c2526e0415ea81d9b3d2 not found: ID does not exist"
Oct 30 19:46:21.550359 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.550339    5907 scope.go:122] "RemoveContainer" containerID="f4bb8b956dfa105291eaef426473484403ff188cb259a28d564513c3619069c0"
Oct 30 19:46:21.557955 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.557914    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b1523289-eee0-4b86-a62f-ee110dde58e6-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1523289-eee0-4b86-a62f-ee110dde58e6-metrics-certs\") pod \"b1523289-eee0-4b86-a62f-ee110dde58e6\" (UID: \"b1523289-eee0-4b86-a62f-ee110dde58e6\") "
Oct 30 19:46:21.558476 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.558451    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b1523289-eee0-4b86-a62f-ee110dde58e6-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1523289-eee0-4b86-a62f-ee110dde58e6-maas-api-tls\") pod \"b1523289-eee0-4b86-a62f-ee110dde58e6\" (UID: \"b1523289-eee0-4b86-a62f-ee110dde58e6\") "
Oct 30 19:46:21.559987 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.558563    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b1523289-eee0-4b86-a62f-ee110dde58e6-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1523289-eee0-4b86-a62f-ee110dde58e6-openshift-service-ca\") pod \"b1523289-eee0-4b86-a62f-ee110dde58e6\" (UID: \"b1523289-eee0-4b86-a62f-ee110dde58e6\") "
Oct 30 19:46:21.559987 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.558817    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b1523289-eee0-4b86-a62f-ee110dde58e6-kube-api-access-pmnn6\" (UniqueName: \"kubernetes.io/projected/b1523289-eee0-4b86-a62f-ee110dde58e6-kube-api-access-pmnn6\") pod \"b1523289-eee0-4b86-a62f-ee110dde58e6\" (UID: \"b1523289-eee0-4b86-a62f-ee110dde58e6\") "
Oct 30 19:46:21.559987 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.559831    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1523289-eee0-4b86-a62f-ee110dde58e6-openshift-service-ca" pod "b1523289-eee0-4b86-a62f-ee110dde58e6" (UID: "b1523289-eee0-4b86-a62f-ee110dde58e6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:21.603373 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.601091    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1523289-eee0-4b86-a62f-ee110dde58e6-maas-api-tls" pod "b1523289-eee0-4b86-a62f-ee110dde58e6" (UID: "b1523289-eee0-4b86-a62f-ee110dde58e6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:46:21.621980 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.618153    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1523289-eee0-4b86-a62f-ee110dde58e6-metrics-certs" pod "b1523289-eee0-4b86-a62f-ee110dde58e6" (UID: "b1523289-eee0-4b86-a62f-ee110dde58e6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:46:21.641535 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.638563    5907 scope.go:122] "RemoveContainer" containerID="f4bb8b956dfa105291eaef426473484403ff188cb259a28d564513c3619069c0"
Oct 30 19:46:21.641535 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.638808    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1523289-eee0-4b86-a62f-ee110dde58e6-kube-api-access-pmnn6" pod "b1523289-eee0-4b86-a62f-ee110dde58e6" (UID: "b1523289-eee0-4b86-a62f-ee110dde58e6"). InnerVolumeSpecName "kube-api-access-pmnn6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:21.641766 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:46:21.641536    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4bb8b956dfa105291eaef426473484403ff188cb259a28d564513c3619069c0\": container with ID starting with f4bb8b956dfa105291eaef426473484403ff188cb259a28d564513c3619069c0 not found: ID does not exist" containerID="f4bb8b956dfa105291eaef426473484403ff188cb259a28d564513c3619069c0"
Oct 30 19:46:21.641766 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.641576    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4bb8b956dfa105291eaef426473484403ff188cb259a28d564513c3619069c0"} err="failed to get container status \"f4bb8b956dfa105291eaef426473484403ff188cb259a28d564513c3619069c0\": rpc error: code = NotFound desc = could not find container \"f4bb8b956dfa105291eaef426473484403ff188cb259a28d564513c3619069c0\": container with ID starting with f4bb8b956dfa105291eaef426473484403ff188cb259a28d564513c3619069c0 not found: ID does not exist"
Oct 30 19:46:21.664889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.661060    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmnn6\" (UniqueName: \"kubernetes.io/projected/b1523289-eee0-4b86-a62f-ee110dde58e6-kube-api-access-pmnn6\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:21.664889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.661102    5907 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1523289-eee0-4b86-a62f-ee110dde58e6-metrics-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:21.664889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.661117    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1523289-eee0-4b86-a62f-ee110dde58e6-maas-api-tls\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:21.664889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.661130    5907 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1523289-eee0-4b86-a62f-ee110dde58e6-openshift-service-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:21.759272 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.702926    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-ptkd4"]
Oct 30 19:46:21.784310 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.784228    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67953efd-ptkd4"]
Oct 30 19:46:21.860903 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.854990    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-67953efd-956bfdb9-4ftfp"]
Oct 30 19:46:21.860903 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.858421    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-67953efd-956bfdb9-4ftfp"]
Oct 30 19:46:21.904724 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.904661    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-3add95-864947fbf5-59nqp"]
Oct 30 19:46:21.907815 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:21.907793    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-3add95-864947fbf5-59nqp"]
Oct 30 19:46:22.289194 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:22.289095    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="364ccf73-2cd0-4922-9460-daca3b6f44e1" path="/var/lib/kubelet/pods/364ccf73-2cd0-4922-9460-daca3b6f44e1/volumes"
Oct 30 19:46:22.289754 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:22.289729    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7e2cfaea-36de-4c52-a106-7b20ffda86e3" path="/var/lib/kubelet/pods/7e2cfaea-36de-4c52-a106-7b20ffda86e3/volumes"
Oct 30 19:46:22.290260 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:22.290239    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b1523289-eee0-4b86-a62f-ee110dde58e6" path="/var/lib/kubelet/pods/b1523289-eee0-4b86-a62f-ee110dde58e6/volumes"
Oct 30 19:46:22.379347 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:22.379289    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3add95-z49q7"]
Oct 30 19:46:22.382795 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:22.382766    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3add95-z49q7"]
Oct 30 19:46:22.464181 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:22.464154    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4b4c34-jccpb"]
Oct 30 19:46:22.464824 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:22.464812    5907 cpu_manager.go:414] "containerMap: removing container" podUID="b1523289-eee0-4b86-a62f-ee110dde58e6" containerName="maas-api"
Oct 30 19:46:22.464884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:22.464827    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b1523289-eee0-4b86-a62f-ee110dde58e6" containerName="maas-api"
Oct 30 19:46:22.464939 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:22.464896    5907 cpu_manager.go:414] "containerMap: removing container" podUID="364ccf73-2cd0-4922-9460-daca3b6f44e1" containerName="maas-api"
Oct 30 19:46:22.464939 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:22.464902    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="364ccf73-2cd0-4922-9460-daca3b6f44e1" containerName="maas-api"
Oct 30 19:46:22.465013 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:22.464996    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="364ccf73-2cd0-4922-9460-daca3b6f44e1" containerName="maas-api"
Oct 30 19:46:22.465013 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:22.465008    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="b1523289-eee0-4b86-a62f-ee110dde58e6" containerName="maas-api"
Oct 30 19:46:22.469754 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:22.469732    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4b4c34-jccpb"
Oct 30 19:46:22.489249 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:22.489229    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4b4c34-jccpb"]
Oct 30 19:46:22.549064 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:22.548899    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a47e70e-xrzvf" event={"ID":"b9d6296e-59ef-4853-8470-d851752d8694","Type":"ContainerStarted","Data":"32d076ef3ea2298b44c2d508ae1696f44fa46bc29914f84fbe2ab87735c240f5"}
Oct 30 19:46:22.550911 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:22.550852    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-57e671e1-fjvwq" event={"ID":"9dd73dbe-af1c-4d56-82e3-228644efda7a","Type":"ContainerStarted","Data":"9ad730b8879252193306fda0400eb5bc2fb3dc31dd165c70286d7e48dc7184e0"}
Oct 30 19:46:22.551016 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:22.550933    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-57e671e1-fjvwq" event={"ID":"9dd73dbe-af1c-4d56-82e3-228644efda7a","Type":"ContainerStarted","Data":"cc26acf30dedcff8095c7ecc338aac39b3e35ed0d2fb5ebb6c5325e3fa57fe2b"}
Oct 30 19:46:22.582569 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:22.582535    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4a53e23a-fa9d-43f9-a651-baa25612b463-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-4b4c34-jccpb\" (UID: \"4a53e23a-fa9d-43f9-a651-baa25612b463\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4b4c34-jccpb"
Oct 30 19:46:22.582672 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:22.582632    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4a53e23a-fa9d-43f9-a651-baa25612b463-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-4b4c34-jccpb\" (UID: \"4a53e23a-fa9d-43f9-a651-baa25612b463\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4b4c34-jccpb"
Oct 30 19:46:22.582672 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:22.582659    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qtn7\" (UniqueName: \"kubernetes.io/projected/4a53e23a-fa9d-43f9-a651-baa25612b463-kube-api-access-6qtn7\") pod \"maas-api-revoke-keys-e2e-worker-w0-4b4c34-jccpb\" (UID: \"4a53e23a-fa9d-43f9-a651-baa25612b463\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4b4c34-jccpb"
Oct 30 19:46:22.683809 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:22.683777    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4a53e23a-fa9d-43f9-a651-baa25612b463-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-4b4c34-jccpb\" (UID: \"4a53e23a-fa9d-43f9-a651-baa25612b463\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4b4c34-jccpb"
Oct 30 19:46:22.684226 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:22.683824    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qtn7\" (UniqueName: \"kubernetes.io/projected/4a53e23a-fa9d-43f9-a651-baa25612b463-kube-api-access-6qtn7\") pod \"maas-api-revoke-keys-e2e-worker-w0-4b4c34-jccpb\" (UID: \"4a53e23a-fa9d-43f9-a651-baa25612b463\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4b4c34-jccpb"
Oct 30 19:46:22.684226 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:22.684121    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4a53e23a-fa9d-43f9-a651-baa25612b463-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-4b4c34-jccpb\" (UID: \"4a53e23a-fa9d-43f9-a651-baa25612b463\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4b4c34-jccpb"
Oct 30 19:46:22.684648 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:22.684624    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4a53e23a-fa9d-43f9-a651-baa25612b463-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-4b4c34-jccpb\" (UID: \"4a53e23a-fa9d-43f9-a651-baa25612b463\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4b4c34-jccpb"
Oct 30 19:46:22.686416 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:22.686392    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4a53e23a-fa9d-43f9-a651-baa25612b463-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-4b4c34-jccpb\" (UID: \"4a53e23a-fa9d-43f9-a651-baa25612b463\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4b4c34-jccpb"
Oct 30 19:46:22.705823 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:22.705792    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qtn7\" (UniqueName: \"kubernetes.io/projected/4a53e23a-fa9d-43f9-a651-baa25612b463-kube-api-access-6qtn7\") pod \"maas-api-revoke-keys-e2e-worker-w0-4b4c34-jccpb\" (UID: \"4a53e23a-fa9d-43f9-a651-baa25612b463\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4b4c34-jccpb"
Oct 30 19:46:22.784431 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:22.784401    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4b4c34-jccpb"
Oct 30 19:46:22.981820 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:22.981782    5907 patch_prober.go:29] interesting pod/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:46:22.982045 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:22.981849    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv" podUID="57838872-c482-41ec-bda0-4a517fb81dcd" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:46:23.103141 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:46:23.103100    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a53e23a_fa9d_43f9_a651_baa25612b463.slice/crio-ffeb568b21067ceff7ab5cba57e9b07b014b998d242f66dce583921780d1067b WatchSource:0}: Error finding container ffeb568b21067ceff7ab5cba57e9b07b014b998d242f66dce583921780d1067b: Status 404 returned error can't find the container with id ffeb568b21067ceff7ab5cba57e9b07b014b998d242f66dce583921780d1067b
Oct 30 19:46:23.103280 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:23.103147    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4b4c34-jccpb"]
Oct 30 19:46:23.561641 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:23.561606    5907 generic.go:356] "Generic (PLEG): container finished" podID="b9d6296e-59ef-4853-8470-d851752d8694" containerID="32d076ef3ea2298b44c2d508ae1696f44fa46bc29914f84fbe2ab87735c240f5" exitCode=0
Oct 30 19:46:23.561818 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:23.561684    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a47e70e-xrzvf" event={"ID":"b9d6296e-59ef-4853-8470-d851752d8694","Type":"ContainerDied","Data":"32d076ef3ea2298b44c2d508ae1696f44fa46bc29914f84fbe2ab87735c240f5"}
Oct 30 19:46:23.563800 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:23.563776    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4b4c34-jccpb" event={"ID":"4a53e23a-fa9d-43f9-a651-baa25612b463","Type":"ContainerStarted","Data":"a757bde9aef867d49a4b10aafa37d80eb77f56c84dc1b6a47ecfe87f7df9a7fb"}
Oct 30 19:46:23.563932 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:23.563824    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4b4c34-jccpb" event={"ID":"4a53e23a-fa9d-43f9-a651-baa25612b463","Type":"ContainerStarted","Data":"ffeb568b21067ceff7ab5cba57e9b07b014b998d242f66dce583921780d1067b"}
Oct 30 19:46:23.565655 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:23.565633    5907 generic.go:356] "Generic (PLEG): container finished" podID="9dd73dbe-af1c-4d56-82e3-228644efda7a" containerID="9ad730b8879252193306fda0400eb5bc2fb3dc31dd165c70286d7e48dc7184e0" exitCode=0
Oct 30 19:46:23.565758 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:23.565716    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-57e671e1-fjvwq" event={"ID":"9dd73dbe-af1c-4d56-82e3-228644efda7a","Type":"ContainerDied","Data":"9ad730b8879252193306fda0400eb5bc2fb3dc31dd165c70286d7e48dc7184e0"}
Oct 30 19:46:23.608083 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:23.608026    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4b4c34-jccpb" podStartSLOduration=1.6080103829999999 podStartE2EDuration="1.608010383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:46:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:46:23.603989291 +0000 UTC m=+1495.446921239" watchObservedRunningTime="2026-10-30 19:46:23.608010383 +0000 UTC m=+1495.450942320"
Oct 30 19:46:23.741716 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:23.741663    5907 patch_prober.go:29] interesting pod/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:46:23.742490 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:23.742457    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c" podUID="81eea74f-1fd9-41b0-8ecf-0501402c6a66" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:46:23.804385 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:23.804360    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a47e70e-xrzvf"
Oct 30 19:46:23.808326 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:23.808305    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-57e671e1-fjvwq"
Oct 30 19:46:23.904855 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:23.904816    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b9d6296e-59ef-4853-8470-d851752d8694-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b9d6296e-59ef-4853-8470-d851752d8694-maas-api-xks-ca\") pod \"b9d6296e-59ef-4853-8470-d851752d8694\" (UID: \"b9d6296e-59ef-4853-8470-d851752d8694\") "
Oct 30 19:46:23.905102 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:23.905073    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9dd73dbe-af1c-4d56-82e3-228644efda7a-kube-api-access-dvcc2\" (UniqueName: \"kubernetes.io/projected/9dd73dbe-af1c-4d56-82e3-228644efda7a-kube-api-access-dvcc2\") pod \"9dd73dbe-af1c-4d56-82e3-228644efda7a\" (UID: \"9dd73dbe-af1c-4d56-82e3-228644efda7a\") "
Oct 30 19:46:23.905244 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:23.905130    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b9d6296e-59ef-4853-8470-d851752d8694-kube-api-access-p7t7r\" (UniqueName: \"kubernetes.io/projected/b9d6296e-59ef-4853-8470-d851752d8694-kube-api-access-p7t7r\") pod \"b9d6296e-59ef-4853-8470-d851752d8694\" (UID: \"b9d6296e-59ef-4853-8470-d851752d8694\") "
Oct 30 19:46:23.905244 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:23.905220    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b9d6296e-59ef-4853-8470-d851752d8694-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b9d6296e-59ef-4853-8470-d851752d8694-maas-api-openshift-ca\") pod \"b9d6296e-59ef-4853-8470-d851752d8694\" (UID: \"b9d6296e-59ef-4853-8470-d851752d8694\") "
Oct 30 19:46:23.905362 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:23.905272    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9dd73dbe-af1c-4d56-82e3-228644efda7a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9dd73dbe-af1c-4d56-82e3-228644efda7a-maas-api-xks-ca\") pod \"9dd73dbe-af1c-4d56-82e3-228644efda7a\" (UID: \"9dd73dbe-af1c-4d56-82e3-228644efda7a\") "
Oct 30 19:46:23.905362 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:23.905329    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9dd73dbe-af1c-4d56-82e3-228644efda7a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9dd73dbe-af1c-4d56-82e3-228644efda7a-maas-api-openshift-ca\") pod \"9dd73dbe-af1c-4d56-82e3-228644efda7a\" (UID: \"9dd73dbe-af1c-4d56-82e3-228644efda7a\") "
Oct 30 19:46:23.905916 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:23.905843    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9d6296e-59ef-4853-8470-d851752d8694-maas-api-openshift-ca" pod "b9d6296e-59ef-4853-8470-d851752d8694" (UID: "b9d6296e-59ef-4853-8470-d851752d8694"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:23.906180 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:23.906148    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b9d6296e-59ef-4853-8470-d851752d8694-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:23.907076 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:23.907043    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9dd73dbe-af1c-4d56-82e3-228644efda7a-maas-api-openshift-ca" pod "9dd73dbe-af1c-4d56-82e3-228644efda7a" (UID: "9dd73dbe-af1c-4d56-82e3-228644efda7a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:23.909723 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:23.909700    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9d6296e-59ef-4853-8470-d851752d8694-maas-api-xks-ca" pod "b9d6296e-59ef-4853-8470-d851752d8694" (UID: "b9d6296e-59ef-4853-8470-d851752d8694"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:46:23.915252 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:23.915227    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9dd73dbe-af1c-4d56-82e3-228644efda7a-maas-api-xks-ca" pod "9dd73dbe-af1c-4d56-82e3-228644efda7a" (UID: "9dd73dbe-af1c-4d56-82e3-228644efda7a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:46:23.915393 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:23.915370    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dd73dbe-af1c-4d56-82e3-228644efda7a-kube-api-access-dvcc2" pod "9dd73dbe-af1c-4d56-82e3-228644efda7a" (UID: "9dd73dbe-af1c-4d56-82e3-228644efda7a"). InnerVolumeSpecName "kube-api-access-dvcc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:23.915586 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:23.915563    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9d6296e-59ef-4853-8470-d851752d8694-kube-api-access-p7t7r" pod "b9d6296e-59ef-4853-8470-d851752d8694" (UID: "b9d6296e-59ef-4853-8470-d851752d8694"). InnerVolumeSpecName "kube-api-access-p7t7r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:23.939811 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:23.939774    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-67953efd-956bfdb9-4ftfp" podUID="364ccf73-2cd0-4922-9460-daca3b6f44e1" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.222:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:46:23.986967 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:23.986895    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-67953efd-b8797cccd-pr7bp" podUID="5dfc7248-d1a7-4093-a2b8-eaf603d9a660" containerName="payload-processing" containerID="cri-o://494cdc5485790e982281a5fde03c47f87a4c17e3cdaf382d37d347e167732b60" gracePeriod=25
Oct 30 19:46:24.005075 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.005046    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ef9a5a-796b5cf9dc-kmcg7" podUID="3b797e1f-bd98-4740-b15d-6ee4e1029d2a" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:46:24.005202 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.005071    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w6-ef9a5a-59fc6bf976-zfzks" podUID="09920b13-483e-47d3-89f0-c22163568979" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:46:24.007875 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.007839    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvcc2\" (UniqueName: \"kubernetes.io/projected/9dd73dbe-af1c-4d56-82e3-228644efda7a-kube-api-access-dvcc2\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:24.007973 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.007883    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7t7r\" (UniqueName: \"kubernetes.io/projected/b9d6296e-59ef-4853-8470-d851752d8694-kube-api-access-p7t7r\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:24.007973 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.007900    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9dd73dbe-af1c-4d56-82e3-228644efda7a-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:24.007973 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.007912    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9dd73dbe-af1c-4d56-82e3-228644efda7a-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:24.007973 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.007924    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b9d6296e-59ef-4853-8470-d851752d8694-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:24.021430 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.021406    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-mhv64" podUID="6c3bc483-1c8c-43d7-a336-ee6e04eda24b" containerName="payload-pre-processing" containerID="cri-o://1e9e1780e07cffd6e6cde29fd932f0dd4116fcbd121eca93ec309b19abb8f4e3" gracePeriod=25
Oct 30 19:46:24.294891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.294836    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2314a32a-7546-45fd-966d-78e512d335f5" path="/var/lib/kubelet/pods/2314a32a-7546-45fd-966d-78e512d335f5/volumes"
Oct 30 19:46:24.329492 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.329464    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-67953efd-b8797cccd-pr7bp"
Oct 30 19:46:24.369238 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.369206    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-mhv64"
Oct 30 19:46:24.425790 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.425758    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6c3bc483-1c8c-43d7-a336-ee6e04eda24b-kube-api-access-pnx6d\" (UniqueName: \"kubernetes.io/projected/6c3bc483-1c8c-43d7-a336-ee6e04eda24b-kube-api-access-pnx6d\") pod \"6c3bc483-1c8c-43d7-a336-ee6e04eda24b\" (UID: \"6c3bc483-1c8c-43d7-a336-ee6e04eda24b\") "
Oct 30 19:46:24.425790 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.425796    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5dfc7248-d1a7-4093-a2b8-eaf603d9a660-kube-api-access-vdhfv\" (UniqueName: \"kubernetes.io/projected/5dfc7248-d1a7-4093-a2b8-eaf603d9a660-kube-api-access-vdhfv\") pod \"5dfc7248-d1a7-4093-a2b8-eaf603d9a660\" (UID: \"5dfc7248-d1a7-4093-a2b8-eaf603d9a660\") "
Oct 30 19:46:24.426060 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.425850    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6c3bc483-1c8c-43d7-a336-ee6e04eda24b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c3bc483-1c8c-43d7-a336-ee6e04eda24b-plugins-config-volume\") pod \"6c3bc483-1c8c-43d7-a336-ee6e04eda24b\" (UID: \"6c3bc483-1c8c-43d7-a336-ee6e04eda24b\") "
Oct 30 19:46:24.426060 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.425963    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5dfc7248-d1a7-4093-a2b8-eaf603d9a660-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5dfc7248-d1a7-4093-a2b8-eaf603d9a660-plugins-config-volume\") pod \"5dfc7248-d1a7-4093-a2b8-eaf603d9a660\" (UID: \"5dfc7248-d1a7-4093-a2b8-eaf603d9a660\") "
Oct 30 19:46:24.426331 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.426302    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c3bc483-1c8c-43d7-a336-ee6e04eda24b-plugins-config-volume" pod "6c3bc483-1c8c-43d7-a336-ee6e04eda24b" (UID: "6c3bc483-1c8c-43d7-a336-ee6e04eda24b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:24.426466 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.426398    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dfc7248-d1a7-4093-a2b8-eaf603d9a660-plugins-config-volume" pod "5dfc7248-d1a7-4093-a2b8-eaf603d9a660" (UID: "5dfc7248-d1a7-4093-a2b8-eaf603d9a660"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:24.426834 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.426807    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c3bc483-1c8c-43d7-a336-ee6e04eda24b-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:24.426972 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.426836    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5dfc7248-d1a7-4093-a2b8-eaf603d9a660-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:24.428772 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.428747    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dfc7248-d1a7-4093-a2b8-eaf603d9a660-kube-api-access-vdhfv" pod "5dfc7248-d1a7-4093-a2b8-eaf603d9a660" (UID: "5dfc7248-d1a7-4093-a2b8-eaf603d9a660"). InnerVolumeSpecName "kube-api-access-vdhfv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:24.429617 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.429594    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c3bc483-1c8c-43d7-a336-ee6e04eda24b-kube-api-access-pnx6d" pod "6c3bc483-1c8c-43d7-a336-ee6e04eda24b" (UID: "6c3bc483-1c8c-43d7-a336-ee6e04eda24b"). InnerVolumeSpecName "kube-api-access-pnx6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:24.528004 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.527925    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnx6d\" (UniqueName: \"kubernetes.io/projected/6c3bc483-1c8c-43d7-a336-ee6e04eda24b-kube-api-access-pnx6d\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:24.528004 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.527951    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vdhfv\" (UniqueName: \"kubernetes.io/projected/5dfc7248-d1a7-4093-a2b8-eaf603d9a660-kube-api-access-vdhfv\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:24.574747 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.574709    5907 generic.go:356] "Generic (PLEG): container finished" podID="5dfc7248-d1a7-4093-a2b8-eaf603d9a660" containerID="494cdc5485790e982281a5fde03c47f87a4c17e3cdaf382d37d347e167732b60" exitCode=0
Oct 30 19:46:24.574913 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.574762    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-67953efd-b8797cccd-pr7bp"
Oct 30 19:46:24.574913 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.574786    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-67953efd-b8797cccd-pr7bp" event={"ID":"5dfc7248-d1a7-4093-a2b8-eaf603d9a660","Type":"ContainerDied","Data":"494cdc5485790e982281a5fde03c47f87a4c17e3cdaf382d37d347e167732b60"}
Oct 30 19:46:24.574913 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.574838    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-67953efd-b8797cccd-pr7bp" event={"ID":"5dfc7248-d1a7-4093-a2b8-eaf603d9a660","Type":"ContainerDied","Data":"4d7963ccf3e6f4fe59a86ff2b79c4b201fe23c87516ffeb68336b94774bbf63c"}
Oct 30 19:46:24.574913 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.574875    5907 scope.go:122] "RemoveContainer" containerID="494cdc5485790e982281a5fde03c47f87a4c17e3cdaf382d37d347e167732b60"
Oct 30 19:46:24.577051 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.577028    5907 generic.go:356] "Generic (PLEG): container finished" podID="4a53e23a-fa9d-43f9-a651-baa25612b463" containerID="a757bde9aef867d49a4b10aafa37d80eb77f56c84dc1b6a47ecfe87f7df9a7fb" exitCode=0
Oct 30 19:46:24.577143 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.577111    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4b4c34-jccpb" event={"ID":"4a53e23a-fa9d-43f9-a651-baa25612b463","Type":"ContainerDied","Data":"a757bde9aef867d49a4b10aafa37d80eb77f56c84dc1b6a47ecfe87f7df9a7fb"}
Oct 30 19:46:24.579048 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.579021    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-57e671e1-fjvwq" event={"ID":"9dd73dbe-af1c-4d56-82e3-228644efda7a","Type":"ContainerDied","Data":"cc26acf30dedcff8095c7ecc338aac39b3e35ed0d2fb5ebb6c5325e3fa57fe2b"}
Oct 30 19:46:24.579165 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.579053    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-57e671e1-fjvwq"
Oct 30 19:46:24.579165 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.579072    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cc26acf30dedcff8095c7ecc338aac39b3e35ed0d2fb5ebb6c5325e3fa57fe2b"
Oct 30 19:46:24.580780 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.580755    5907 generic.go:356] "Generic (PLEG): container finished" podID="6c3bc483-1c8c-43d7-a336-ee6e04eda24b" containerID="1e9e1780e07cffd6e6cde29fd932f0dd4116fcbd121eca93ec309b19abb8f4e3" exitCode=0
Oct 30 19:46:24.580780 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.580786    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-mhv64"
Oct 30 19:46:24.580974 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.580839    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-mhv64" event={"ID":"6c3bc483-1c8c-43d7-a336-ee6e04eda24b","Type":"ContainerDied","Data":"1e9e1780e07cffd6e6cde29fd932f0dd4116fcbd121eca93ec309b19abb8f4e3"}
Oct 30 19:46:24.580974 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.580920    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-mhv64" event={"ID":"6c3bc483-1c8c-43d7-a336-ee6e04eda24b","Type":"ContainerDied","Data":"a418ef2c8351cf28071de2f39b9590c5a10e65258617c073b85b5a1ac7785b80"}
Oct 30 19:46:24.585002 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.584910    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a47e70e-xrzvf" event={"ID":"b9d6296e-59ef-4853-8470-d851752d8694","Type":"ContainerDied","Data":"6f25b3f2a29e45c5c222a9d9fd15b24f2e20ce031690d428b73c780d63904c3e"}
Oct 30 19:46:24.585137 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.585118    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6f25b3f2a29e45c5c222a9d9fd15b24f2e20ce031690d428b73c780d63904c3e"
Oct 30 19:46:24.585280 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.585266    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7a47e70e-xrzvf"
Oct 30 19:46:24.590897 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.590876    5907 scope.go:122] "RemoveContainer" containerID="494cdc5485790e982281a5fde03c47f87a4c17e3cdaf382d37d347e167732b60"
Oct 30 19:46:24.591302 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:46:24.591273    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"494cdc5485790e982281a5fde03c47f87a4c17e3cdaf382d37d347e167732b60\": container with ID starting with 494cdc5485790e982281a5fde03c47f87a4c17e3cdaf382d37d347e167732b60 not found: ID does not exist" containerID="494cdc5485790e982281a5fde03c47f87a4c17e3cdaf382d37d347e167732b60"
Oct 30 19:46:24.591393 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.591303    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"494cdc5485790e982281a5fde03c47f87a4c17e3cdaf382d37d347e167732b60"} err="failed to get container status \"494cdc5485790e982281a5fde03c47f87a4c17e3cdaf382d37d347e167732b60\": rpc error: code = NotFound desc = could not find container \"494cdc5485790e982281a5fde03c47f87a4c17e3cdaf382d37d347e167732b60\": container with ID starting with 494cdc5485790e982281a5fde03c47f87a4c17e3cdaf382d37d347e167732b60 not found: ID does not exist"
Oct 30 19:46:24.591393 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.591324    5907 scope.go:122] "RemoveContainer" containerID="1e9e1780e07cffd6e6cde29fd932f0dd4116fcbd121eca93ec309b19abb8f4e3"
Oct 30 19:46:24.601631 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.601608    5907 scope.go:122] "RemoveContainer" containerID="1e9e1780e07cffd6e6cde29fd932f0dd4116fcbd121eca93ec309b19abb8f4e3"
Oct 30 19:46:24.601945 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:46:24.601923    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e9e1780e07cffd6e6cde29fd932f0dd4116fcbd121eca93ec309b19abb8f4e3\": container with ID starting with 1e9e1780e07cffd6e6cde29fd932f0dd4116fcbd121eca93ec309b19abb8f4e3 not found: ID does not exist" containerID="1e9e1780e07cffd6e6cde29fd932f0dd4116fcbd121eca93ec309b19abb8f4e3"
Oct 30 19:46:24.602006 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.601953    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e9e1780e07cffd6e6cde29fd932f0dd4116fcbd121eca93ec309b19abb8f4e3"} err="failed to get container status \"1e9e1780e07cffd6e6cde29fd932f0dd4116fcbd121eca93ec309b19abb8f4e3\": rpc error: code = NotFound desc = could not find container \"1e9e1780e07cffd6e6cde29fd932f0dd4116fcbd121eca93ec309b19abb8f4e3\": container with ID starting with 1e9e1780e07cffd6e6cde29fd932f0dd4116fcbd121eca93ec309b19abb8f4e3 not found: ID does not exist"
Oct 30 19:46:24.632313 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.632286    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-mhv64"]
Oct 30 19:46:24.637417 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.637394    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-67953efd-6797bd4db7-mhv64"]
Oct 30 19:46:24.673519 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.673487    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-67953efd-b8797cccd-pr7bp"]
Oct 30 19:46:24.678251 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:24.678229    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-67953efd-b8797cccd-pr7bp"]
Oct 30 19:46:25.811138 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:25.811111    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4b4c34-jccpb"
Oct 30 19:46:25.845647 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:25.845617    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4a53e23a-fa9d-43f9-a651-baa25612b463-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4a53e23a-fa9d-43f9-a651-baa25612b463-maas-api-xks-ca\") pod \"4a53e23a-fa9d-43f9-a651-baa25612b463\" (UID: \"4a53e23a-fa9d-43f9-a651-baa25612b463\") "
Oct 30 19:46:25.845812 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:25.845664    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4a53e23a-fa9d-43f9-a651-baa25612b463-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4a53e23a-fa9d-43f9-a651-baa25612b463-maas-api-openshift-ca\") pod \"4a53e23a-fa9d-43f9-a651-baa25612b463\" (UID: \"4a53e23a-fa9d-43f9-a651-baa25612b463\") "
Oct 30 19:46:25.845882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:25.845835    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4a53e23a-fa9d-43f9-a651-baa25612b463-kube-api-access-6qtn7\" (UniqueName: \"kubernetes.io/projected/4a53e23a-fa9d-43f9-a651-baa25612b463-kube-api-access-6qtn7\") pod \"4a53e23a-fa9d-43f9-a651-baa25612b463\" (UID: \"4a53e23a-fa9d-43f9-a651-baa25612b463\") "
Oct 30 19:46:25.846107 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:25.846076    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a53e23a-fa9d-43f9-a651-baa25612b463-maas-api-openshift-ca" pod "4a53e23a-fa9d-43f9-a651-baa25612b463" (UID: "4a53e23a-fa9d-43f9-a651-baa25612b463"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:25.846504 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:25.846485    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4a53e23a-fa9d-43f9-a651-baa25612b463-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:25.848029 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:25.848009    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a53e23a-fa9d-43f9-a651-baa25612b463-kube-api-access-6qtn7" pod "4a53e23a-fa9d-43f9-a651-baa25612b463" (UID: "4a53e23a-fa9d-43f9-a651-baa25612b463"). InnerVolumeSpecName "kube-api-access-6qtn7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:25.848106 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:25.848093    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a53e23a-fa9d-43f9-a651-baa25612b463-maas-api-xks-ca" pod "4a53e23a-fa9d-43f9-a651-baa25612b463" (UID: "4a53e23a-fa9d-43f9-a651-baa25612b463"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:46:25.947801 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:25.947767    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4a53e23a-fa9d-43f9-a651-baa25612b463-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:25.947801 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:25.947796    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qtn7\" (UniqueName: \"kubernetes.io/projected/4a53e23a-fa9d-43f9-a651-baa25612b463-kube-api-access-6qtn7\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:26.288805 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.288766    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5dfc7248-d1a7-4093-a2b8-eaf603d9a660" path="/var/lib/kubelet/pods/5dfc7248-d1a7-4093-a2b8-eaf603d9a660/volumes"
Oct 30 19:46:26.289333 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.289309    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6c3bc483-1c8c-43d7-a336-ee6e04eda24b" path="/var/lib/kubelet/pods/6c3bc483-1c8c-43d7-a336-ee6e04eda24b/volumes"
Oct 30 19:46:26.617022 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.616851    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4b4c34-jccpb"
Oct 30 19:46:26.617022 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.616940    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4b4c34-jccpb" event={"ID":"4a53e23a-fa9d-43f9-a651-baa25612b463","Type":"ContainerDied","Data":"ffeb568b21067ceff7ab5cba57e9b07b014b998d242f66dce583921780d1067b"}
Oct 30 19:46:26.617022 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.616995    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ffeb568b21067ceff7ab5cba57e9b07b014b998d242f66dce583921780d1067b"
Oct 30 19:46:26.620477 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.620364    5907 generic.go:356] "Generic (PLEG): container finished" podID="57838872-c482-41ec-bda0-4a517fb81dcd" containerID="7d5cbf1113d7cdb51c3e45581ca40732e044dbc3b8d173681cf18aedc4290c97" exitCode=0
Oct 30 19:46:26.620604 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.620517    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv" event={"ID":"57838872-c482-41ec-bda0-4a517fb81dcd","Type":"ContainerDied","Data":"7d5cbf1113d7cdb51c3e45581ca40732e044dbc3b8d173681cf18aedc4290c97"}
Oct 30 19:46:26.620604 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.620568    5907 scope.go:122] "RemoveContainer" containerID="1d1433dd49d5c61722711f2b46da6b97cb8aa4b0fc9e164520f3452b073521af"
Oct 30 19:46:26.622429 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.621917    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:46:26.631202 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.624851    5907 generic.go:356] "Generic (PLEG): container finished" podID="282f4fd8-512c-41b9-9989-0b84f2de1ce5" containerID="269422201e7a715b94c4696784ba9bc6fc8a305ce1ef1a2aef55601d94590ecf" exitCode=0
Oct 30 19:46:26.631202 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.625109    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc" event={"ID":"282f4fd8-512c-41b9-9989-0b84f2de1ce5","Type":"ContainerDied","Data":"269422201e7a715b94c4696784ba9bc6fc8a305ce1ef1a2aef55601d94590ecf"}
Oct 30 19:46:26.631202 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.627050    5907 generic.go:356] "Generic (PLEG): container finished" podID="81eea74f-1fd9-41b0-8ecf-0501402c6a66" containerID="5586215f406d71926acf24937a198f86cc4e4be5e19c8e9e111c314a4823a94d" exitCode=0
Oct 30 19:46:26.631202 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.627103    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c" event={"ID":"81eea74f-1fd9-41b0-8ecf-0501402c6a66","Type":"ContainerDied","Data":"5586215f406d71926acf24937a198f86cc4e4be5e19c8e9e111c314a4823a94d"}
Oct 30 19:46:26.668972 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.668938    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/57838872-c482-41ec-bda0-4a517fb81dcd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57838872-c482-41ec-bda0-4a517fb81dcd-istio-envoy\") pod \"57838872-c482-41ec-bda0-4a517fb81dcd\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") "
Oct 30 19:46:26.669132 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.669093    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/57838872-c482-41ec-bda0-4a517fb81dcd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57838872-c482-41ec-bda0-4a517fb81dcd-istiod-ca-cert\") pod \"57838872-c482-41ec-bda0-4a517fb81dcd\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") "
Oct 30 19:46:26.669196 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.669150    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/57838872-c482-41ec-bda0-4a517fb81dcd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57838872-c482-41ec-bda0-4a517fb81dcd-workload-certs\") pod \"57838872-c482-41ec-bda0-4a517fb81dcd\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") "
Oct 30 19:46:26.671412 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.669252    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/57838872-c482-41ec-bda0-4a517fb81dcd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57838872-c482-41ec-bda0-4a517fb81dcd-istio-data\") pod \"57838872-c482-41ec-bda0-4a517fb81dcd\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") "
Oct 30 19:46:26.671412 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.669285    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/57838872-c482-41ec-bda0-4a517fb81dcd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57838872-c482-41ec-bda0-4a517fb81dcd-istio-podinfo\") pod \"57838872-c482-41ec-bda0-4a517fb81dcd\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") "
Oct 30 19:46:26.671412 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.669350    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/57838872-c482-41ec-bda0-4a517fb81dcd-kube-api-access-f9dl4\" (UniqueName: \"kubernetes.io/projected/57838872-c482-41ec-bda0-4a517fb81dcd-kube-api-access-f9dl4\") pod \"57838872-c482-41ec-bda0-4a517fb81dcd\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") "
Oct 30 19:46:26.671412 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.669458    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/57838872-c482-41ec-bda0-4a517fb81dcd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57838872-c482-41ec-bda0-4a517fb81dcd-credential-socket\") pod \"57838872-c482-41ec-bda0-4a517fb81dcd\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") "
Oct 30 19:46:26.671412 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.669495    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/57838872-c482-41ec-bda0-4a517fb81dcd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57838872-c482-41ec-bda0-4a517fb81dcd-workload-socket\") pod \"57838872-c482-41ec-bda0-4a517fb81dcd\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") "
Oct 30 19:46:26.671412 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.669571    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/57838872-c482-41ec-bda0-4a517fb81dcd-istio-token\" (UniqueName: \"kubernetes.io/projected/57838872-c482-41ec-bda0-4a517fb81dcd-istio-token\") pod \"57838872-c482-41ec-bda0-4a517fb81dcd\" (UID: \"57838872-c482-41ec-bda0-4a517fb81dcd\") "
Oct 30 19:46:26.671412 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.670836    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57838872-c482-41ec-bda0-4a517fb81dcd-istiod-ca-cert" pod "57838872-c482-41ec-bda0-4a517fb81dcd" (UID: "57838872-c482-41ec-bda0-4a517fb81dcd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:26.672247 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.671969    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57838872-c482-41ec-bda0-4a517fb81dcd-credential-socket" pod "57838872-c482-41ec-bda0-4a517fb81dcd" (UID: "57838872-c482-41ec-bda0-4a517fb81dcd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:26.672247 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.672120    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57838872-c482-41ec-bda0-4a517fb81dcd-istio-data" pod "57838872-c482-41ec-bda0-4a517fb81dcd" (UID: "57838872-c482-41ec-bda0-4a517fb81dcd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:26.672247 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.672158    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57838872-c482-41ec-bda0-4a517fb81dcd-workload-socket" pod "57838872-c482-41ec-bda0-4a517fb81dcd" (UID: "57838872-c482-41ec-bda0-4a517fb81dcd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:26.672247 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.672203    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57838872-c482-41ec-bda0-4a517fb81dcd-workload-certs" pod "57838872-c482-41ec-bda0-4a517fb81dcd" (UID: "57838872-c482-41ec-bda0-4a517fb81dcd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:26.676658 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.674030    5907 scope.go:122] "RemoveContainer" containerID="01633586736e0aaf824342a7876015a1a00880bfcafb68c4caa6f9f362652fd1"
Oct 30 19:46:26.685397 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.685377    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/57838872-c482-41ec-bda0-4a517fb81dcd-istio-podinfo" pod "57838872-c482-41ec-bda0-4a517fb81dcd" (UID: "57838872-c482-41ec-bda0-4a517fb81dcd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:46:26.687316 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.687288    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57838872-c482-41ec-bda0-4a517fb81dcd-kube-api-access-f9dl4" pod "57838872-c482-41ec-bda0-4a517fb81dcd" (UID: "57838872-c482-41ec-bda0-4a517fb81dcd"). InnerVolumeSpecName "kube-api-access-f9dl4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:26.692138 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.692043    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57838872-c482-41ec-bda0-4a517fb81dcd-istio-envoy" pod "57838872-c482-41ec-bda0-4a517fb81dcd" (UID: "57838872-c482-41ec-bda0-4a517fb81dcd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:26.698490 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.698447    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57838872-c482-41ec-bda0-4a517fb81dcd-istio-token" pod "57838872-c482-41ec-bda0-4a517fb81dcd" (UID: "57838872-c482-41ec-bda0-4a517fb81dcd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:26.761565 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.761507    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:46:26.771829 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.771798    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57838872-c482-41ec-bda0-4a517fb81dcd-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:26.772070 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.771829    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57838872-c482-41ec-bda0-4a517fb81dcd-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:26.772070 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.771847    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57838872-c482-41ec-bda0-4a517fb81dcd-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:26.772070 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.771902    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57838872-c482-41ec-bda0-4a517fb81dcd-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:26.772070 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.771915    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57838872-c482-41ec-bda0-4a517fb81dcd-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:26.772070 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.771925    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57838872-c482-41ec-bda0-4a517fb81dcd-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:26.772070 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.771935    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57838872-c482-41ec-bda0-4a517fb81dcd-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:26.772070 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.771945    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57838872-c482-41ec-bda0-4a517fb81dcd-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:26.772070 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.771956    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9dl4\" (UniqueName: \"kubernetes.io/projected/57838872-c482-41ec-bda0-4a517fb81dcd-kube-api-access-f9dl4\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:26.865599 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.865567    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"
Oct 30 19:46:26.872898 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.872832    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/282f4fd8-512c-41b9-9989-0b84f2de1ce5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/282f4fd8-512c-41b9-9989-0b84f2de1ce5-istio-data\") pod \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") "
Oct 30 19:46:26.873031 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.872913    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/282f4fd8-512c-41b9-9989-0b84f2de1ce5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/282f4fd8-512c-41b9-9989-0b84f2de1ce5-istio-envoy\") pod \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") "
Oct 30 19:46:26.873031 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.872948    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/282f4fd8-512c-41b9-9989-0b84f2de1ce5-kube-api-access-pvsjd\" (UniqueName: \"kubernetes.io/projected/282f4fd8-512c-41b9-9989-0b84f2de1ce5-kube-api-access-pvsjd\") pod \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") "
Oct 30 19:46:26.873153 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.873084    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/282f4fd8-512c-41b9-9989-0b84f2de1ce5-istio-token\" (UniqueName: \"kubernetes.io/projected/282f4fd8-512c-41b9-9989-0b84f2de1ce5-istio-token\") pod \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") "
Oct 30 19:46:26.873153 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.873129    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/282f4fd8-512c-41b9-9989-0b84f2de1ce5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/282f4fd8-512c-41b9-9989-0b84f2de1ce5-istio-podinfo\") pod \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") "
Oct 30 19:46:26.873255 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.873169    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/282f4fd8-512c-41b9-9989-0b84f2de1ce5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/282f4fd8-512c-41b9-9989-0b84f2de1ce5-credential-socket\") pod \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") "
Oct 30 19:46:26.873315 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.873282    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/282f4fd8-512c-41b9-9989-0b84f2de1ce5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/282f4fd8-512c-41b9-9989-0b84f2de1ce5-istiod-ca-cert\") pod \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") "
Oct 30 19:46:26.873364 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.873313    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/282f4fd8-512c-41b9-9989-0b84f2de1ce5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/282f4fd8-512c-41b9-9989-0b84f2de1ce5-workload-socket\") pod \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") "
Oct 30 19:46:26.873638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.873516    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/282f4fd8-512c-41b9-9989-0b84f2de1ce5-credential-socket" pod "282f4fd8-512c-41b9-9989-0b84f2de1ce5" (UID: "282f4fd8-512c-41b9-9989-0b84f2de1ce5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:26.873638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.873524    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/282f4fd8-512c-41b9-9989-0b84f2de1ce5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/282f4fd8-512c-41b9-9989-0b84f2de1ce5-workload-certs\") pod \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\" (UID: \"282f4fd8-512c-41b9-9989-0b84f2de1ce5\") "
Oct 30 19:46:26.873882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.873754    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/282f4fd8-512c-41b9-9989-0b84f2de1ce5-workload-certs" pod "282f4fd8-512c-41b9-9989-0b84f2de1ce5" (UID: "282f4fd8-512c-41b9-9989-0b84f2de1ce5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:26.873882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.873771    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/282f4fd8-512c-41b9-9989-0b84f2de1ce5-istiod-ca-cert" pod "282f4fd8-512c-41b9-9989-0b84f2de1ce5" (UID: "282f4fd8-512c-41b9-9989-0b84f2de1ce5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:26.874654 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.874139    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/282f4fd8-512c-41b9-9989-0b84f2de1ce5-workload-socket" pod "282f4fd8-512c-41b9-9989-0b84f2de1ce5" (UID: "282f4fd8-512c-41b9-9989-0b84f2de1ce5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:26.874654 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.874572    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/282f4fd8-512c-41b9-9989-0b84f2de1ce5-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:26.874654 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.874597    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/282f4fd8-512c-41b9-9989-0b84f2de1ce5-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:26.874654 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.874609    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/282f4fd8-512c-41b9-9989-0b84f2de1ce5-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:26.874654 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.874621    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/282f4fd8-512c-41b9-9989-0b84f2de1ce5-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:26.874926 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.874742    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/282f4fd8-512c-41b9-9989-0b84f2de1ce5-istio-data" pod "282f4fd8-512c-41b9-9989-0b84f2de1ce5" (UID: "282f4fd8-512c-41b9-9989-0b84f2de1ce5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:26.876317 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.876278    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/282f4fd8-512c-41b9-9989-0b84f2de1ce5-kube-api-access-pvsjd" pod "282f4fd8-512c-41b9-9989-0b84f2de1ce5" (UID: "282f4fd8-512c-41b9-9989-0b84f2de1ce5"). InnerVolumeSpecName "kube-api-access-pvsjd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:26.877226 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.877202    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/282f4fd8-512c-41b9-9989-0b84f2de1ce5-istio-token" pod "282f4fd8-512c-41b9-9989-0b84f2de1ce5" (UID: "282f4fd8-512c-41b9-9989-0b84f2de1ce5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:26.878409 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.878375    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/282f4fd8-512c-41b9-9989-0b84f2de1ce5-istio-podinfo" pod "282f4fd8-512c-41b9-9989-0b84f2de1ce5" (UID: "282f4fd8-512c-41b9-9989-0b84f2de1ce5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:46:26.879429 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.879407    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/282f4fd8-512c-41b9-9989-0b84f2de1ce5-istio-envoy" pod "282f4fd8-512c-41b9-9989-0b84f2de1ce5" (UID: "282f4fd8-512c-41b9-9989-0b84f2de1ce5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:26.925908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.922108    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"]
Oct 30 19:46:26.975456 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.975417    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/81eea74f-1fd9-41b0-8ecf-0501402c6a66-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81eea74f-1fd9-41b0-8ecf-0501402c6a66-istiod-ca-cert\") pod \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") "
Oct 30 19:46:26.975602 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.975503    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/81eea74f-1fd9-41b0-8ecf-0501402c6a66-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81eea74f-1fd9-41b0-8ecf-0501402c6a66-istio-podinfo\") pod \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") "
Oct 30 19:46:26.975602 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.975577    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81eea74f-1fd9-41b0-8ecf-0501402c6a66-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81eea74f-1fd9-41b0-8ecf-0501402c6a66-credential-socket\") pod \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") "
Oct 30 19:46:26.975669 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.975631    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81eea74f-1fd9-41b0-8ecf-0501402c6a66-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81eea74f-1fd9-41b0-8ecf-0501402c6a66-workload-certs\") pod \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") "
Oct 30 19:46:26.975720 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.975669    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81eea74f-1fd9-41b0-8ecf-0501402c6a66-kube-api-access-fsr6j\" (UniqueName: \"kubernetes.io/projected/81eea74f-1fd9-41b0-8ecf-0501402c6a66-kube-api-access-fsr6j\") pod \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") "
Oct 30 19:46:26.975759 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.975719    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81eea74f-1fd9-41b0-8ecf-0501402c6a66-istio-token\" (UniqueName: \"kubernetes.io/projected/81eea74f-1fd9-41b0-8ecf-0501402c6a66-istio-token\") pod \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") "
Oct 30 19:46:26.975759 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.975748    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81eea74f-1fd9-41b0-8ecf-0501402c6a66-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81eea74f-1fd9-41b0-8ecf-0501402c6a66-istio-envoy\") pod \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") "
Oct 30 19:46:26.975847 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.975830    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81eea74f-1fd9-41b0-8ecf-0501402c6a66-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81eea74f-1fd9-41b0-8ecf-0501402c6a66-workload-socket\") pod \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") "
Oct 30 19:46:26.976050 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.976030    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81eea74f-1fd9-41b0-8ecf-0501402c6a66-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81eea74f-1fd9-41b0-8ecf-0501402c6a66-istio-data\") pod \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\" (UID: \"81eea74f-1fd9-41b0-8ecf-0501402c6a66\") "
Oct 30 19:46:26.976844 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.976597    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/282f4fd8-512c-41b9-9989-0b84f2de1ce5-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:26.976844 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.976621    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/282f4fd8-512c-41b9-9989-0b84f2de1ce5-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:26.976844 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.976635    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvsjd\" (UniqueName: \"kubernetes.io/projected/282f4fd8-512c-41b9-9989-0b84f2de1ce5-kube-api-access-pvsjd\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:26.976844 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.976648    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/282f4fd8-512c-41b9-9989-0b84f2de1ce5-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:26.976844 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.976661    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/282f4fd8-512c-41b9-9989-0b84f2de1ce5-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:26.976844 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.976683    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81eea74f-1fd9-41b0-8ecf-0501402c6a66-workload-certs" pod "81eea74f-1fd9-41b0-8ecf-0501402c6a66" (UID: "81eea74f-1fd9-41b0-8ecf-0501402c6a66"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:26.977229 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.976902    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81eea74f-1fd9-41b0-8ecf-0501402c6a66-credential-socket" pod "81eea74f-1fd9-41b0-8ecf-0501402c6a66" (UID: "81eea74f-1fd9-41b0-8ecf-0501402c6a66"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:26.977229 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.977132    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81eea74f-1fd9-41b0-8ecf-0501402c6a66-workload-socket" pod "81eea74f-1fd9-41b0-8ecf-0501402c6a66" (UID: "81eea74f-1fd9-41b0-8ecf-0501402c6a66"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:26.977404 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.977372    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81eea74f-1fd9-41b0-8ecf-0501402c6a66-istio-data" pod "81eea74f-1fd9-41b0-8ecf-0501402c6a66" (UID: "81eea74f-1fd9-41b0-8ecf-0501402c6a66"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:26.977464 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.977446    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81eea74f-1fd9-41b0-8ecf-0501402c6a66-istiod-ca-cert" pod "81eea74f-1fd9-41b0-8ecf-0501402c6a66" (UID: "81eea74f-1fd9-41b0-8ecf-0501402c6a66"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:26.979765 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.979725    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81eea74f-1fd9-41b0-8ecf-0501402c6a66-istio-token" pod "81eea74f-1fd9-41b0-8ecf-0501402c6a66" (UID: "81eea74f-1fd9-41b0-8ecf-0501402c6a66"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:26.979982 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.979819    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81eea74f-1fd9-41b0-8ecf-0501402c6a66-kube-api-access-fsr6j" pod "81eea74f-1fd9-41b0-8ecf-0501402c6a66" (UID: "81eea74f-1fd9-41b0-8ecf-0501402c6a66"). InnerVolumeSpecName "kube-api-access-fsr6j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:26.980263 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.980234    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81eea74f-1fd9-41b0-8ecf-0501402c6a66-istio-envoy" pod "81eea74f-1fd9-41b0-8ecf-0501402c6a66" (UID: "81eea74f-1fd9-41b0-8ecf-0501402c6a66"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:26.980398 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:26.980377    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/81eea74f-1fd9-41b0-8ecf-0501402c6a66-istio-podinfo" pod "81eea74f-1fd9-41b0-8ecf-0501402c6a66" (UID: "81eea74f-1fd9-41b0-8ecf-0501402c6a66"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:46:27.078240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:27.078162    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81eea74f-1fd9-41b0-8ecf-0501402c6a66-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:27.078240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:27.078189    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81eea74f-1fd9-41b0-8ecf-0501402c6a66-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:27.078240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:27.078203    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fsr6j\" (UniqueName: \"kubernetes.io/projected/81eea74f-1fd9-41b0-8ecf-0501402c6a66-kube-api-access-fsr6j\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:27.078240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:27.078216    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81eea74f-1fd9-41b0-8ecf-0501402c6a66-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:27.078240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:27.078226    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81eea74f-1fd9-41b0-8ecf-0501402c6a66-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:27.078240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:27.078234    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81eea74f-1fd9-41b0-8ecf-0501402c6a66-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:27.078240 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:27.078242    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81eea74f-1fd9-41b0-8ecf-0501402c6a66-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:27.078587 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:27.078249    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81eea74f-1fd9-41b0-8ecf-0501402c6a66-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:27.078587 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:27.078257    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81eea74f-1fd9-41b0-8ecf-0501402c6a66-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:27.121072 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:27.121039    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"]
Oct 30 19:46:27.635501 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:27.635470    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"
Oct 30 19:46:27.635680 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:27.635508    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv" event={"ID":"57838872-c482-41ec-bda0-4a517fb81dcd","Type":"ContainerDied","Data":"6580af7448b6ee153e9c74962b10eece37c35ade55a9159837fdc0132ccda217"}
Oct 30 19:46:27.635680 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:27.635562    5907 scope.go:122] "RemoveContainer" containerID="7d5cbf1113d7cdb51c3e45581ca40732e044dbc3b8d173681cf18aedc4290c97"
Oct 30 19:46:27.637289 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:27.637270    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"
Oct 30 19:46:27.637408 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:27.637320    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc" event={"ID":"282f4fd8-512c-41b9-9989-0b84f2de1ce5","Type":"ContainerDied","Data":"36c3c91ce9e5ddd221b84e493f009ed8c9763ee444296777ef032292142d0cc3"}
Oct 30 19:46:27.639398 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:27.639269    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"
Oct 30 19:46:27.639398 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:27.639346    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c" event={"ID":"81eea74f-1fd9-41b0-8ecf-0501402c6a66","Type":"ContainerDied","Data":"40b4917b5e42f9e099b8d7eb074d075de95321293d9b2490464781afa778b11a"}
Oct 30 19:46:27.652930 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:27.652890    5907 scope.go:122] "RemoveContainer" containerID="269422201e7a715b94c4696784ba9bc6fc8a305ce1ef1a2aef55601d94590ecf"
Oct 30 19:46:27.664763 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:27.664745    5907 scope.go:122] "RemoveContainer" containerID="5586215f406d71926acf24937a198f86cc4e4be5e19c8e9e111c314a4823a94d"
Oct 30 19:46:27.737149 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:27.737114    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"]
Oct 30 19:46:27.745717 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:27.745692    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-e0e300-openshift-default-7c678b975f-fg4xv"]
Oct 30 19:46:27.795141 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:27.795088    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"]
Oct 30 19:46:27.814340 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:27.814314    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-rj76c"]
Oct 30 19:46:27.858119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:27.858089    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"]
Oct 30 19:46:27.871852 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:27.871808    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-3735b2-openshift-default-69bc97d6f-wrpbc"]
Oct 30 19:46:28.290723 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:28.290688    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="282f4fd8-512c-41b9-9989-0b84f2de1ce5" path="/var/lib/kubelet/pods/282f4fd8-512c-41b9-9989-0b84f2de1ce5/volumes"
Oct 30 19:46:28.291173 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:28.291158    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="57838872-c482-41ec-bda0-4a517fb81dcd" path="/var/lib/kubelet/pods/57838872-c482-41ec-bda0-4a517fb81dcd/volumes"
Oct 30 19:46:28.291579 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:28.291566    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="81eea74f-1fd9-41b0-8ecf-0501402c6a66" path="/var/lib/kubelet/pods/81eea74f-1fd9-41b0-8ecf-0501402c6a66/volumes"
Oct 30 19:46:29.111431 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:29.111406    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-ef9a5a-59fc6bf976-zfzks"
Oct 30 19:46:29.114974 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:29.114956    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ef9a5a-796b5cf9dc-kmcg7"
Oct 30 19:46:29.245916 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:29.244298    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3b797e1f-bd98-4740-b15d-6ee4e1029d2a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b797e1f-bd98-4740-b15d-6ee4e1029d2a-plugins-config-volume\") pod \"3b797e1f-bd98-4740-b15d-6ee4e1029d2a\" (UID: \"3b797e1f-bd98-4740-b15d-6ee4e1029d2a\") "
Oct 30 19:46:29.245916 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:29.244401    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/09920b13-483e-47d3-89f0-c22163568979-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09920b13-483e-47d3-89f0-c22163568979-plugins-config-volume\") pod \"09920b13-483e-47d3-89f0-c22163568979\" (UID: \"09920b13-483e-47d3-89f0-c22163568979\") "
Oct 30 19:46:29.245916 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:29.244635    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/09920b13-483e-47d3-89f0-c22163568979-kube-api-access-jzc2b\" (UniqueName: \"kubernetes.io/projected/09920b13-483e-47d3-89f0-c22163568979-kube-api-access-jzc2b\") pod \"09920b13-483e-47d3-89f0-c22163568979\" (UID: \"09920b13-483e-47d3-89f0-c22163568979\") "
Oct 30 19:46:29.245916 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:29.244694    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3b797e1f-bd98-4740-b15d-6ee4e1029d2a-kube-api-access-rnqk2\" (UniqueName: \"kubernetes.io/projected/3b797e1f-bd98-4740-b15d-6ee4e1029d2a-kube-api-access-rnqk2\") pod \"3b797e1f-bd98-4740-b15d-6ee4e1029d2a\" (UID: \"3b797e1f-bd98-4740-b15d-6ee4e1029d2a\") "
Oct 30 19:46:29.277283 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:29.269169    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b797e1f-bd98-4740-b15d-6ee4e1029d2a-plugins-config-volume" pod "3b797e1f-bd98-4740-b15d-6ee4e1029d2a" (UID: "3b797e1f-bd98-4740-b15d-6ee4e1029d2a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:29.278309 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:29.278269    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09920b13-483e-47d3-89f0-c22163568979-plugins-config-volume" pod "09920b13-483e-47d3-89f0-c22163568979" (UID: "09920b13-483e-47d3-89f0-c22163568979"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:29.286293 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:29.286234    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b797e1f-bd98-4740-b15d-6ee4e1029d2a-kube-api-access-rnqk2" pod "3b797e1f-bd98-4740-b15d-6ee4e1029d2a" (UID: "3b797e1f-bd98-4740-b15d-6ee4e1029d2a"). InnerVolumeSpecName "kube-api-access-rnqk2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:29.288740 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:29.288713    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09920b13-483e-47d3-89f0-c22163568979-kube-api-access-jzc2b" pod "09920b13-483e-47d3-89f0-c22163568979" (UID: "09920b13-483e-47d3-89f0-c22163568979"). InnerVolumeSpecName "kube-api-access-jzc2b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:29.358841 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:29.358755    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzc2b\" (UniqueName: \"kubernetes.io/projected/09920b13-483e-47d3-89f0-c22163568979-kube-api-access-jzc2b\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:29.358841 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:29.358784    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rnqk2\" (UniqueName: \"kubernetes.io/projected/3b797e1f-bd98-4740-b15d-6ee4e1029d2a-kube-api-access-rnqk2\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:29.358841 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:29.358800    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b797e1f-bd98-4740-b15d-6ee4e1029d2a-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:29.358841 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:29.358812    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09920b13-483e-47d3-89f0-c22163568979-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:29.662168 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:29.662136    5907 generic.go:356] "Generic (PLEG): container finished" podID="09920b13-483e-47d3-89f0-c22163568979" containerID="874488b95670c3078fd206e79eb89fb510f58632d62b6a81e4c02f9372f85146" exitCode=0
Oct 30 19:46:29.662372 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:29.662199    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-ef9a5a-59fc6bf976-zfzks"
Oct 30 19:46:29.662372 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:29.662221    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-ef9a5a-59fc6bf976-zfzks" event={"ID":"09920b13-483e-47d3-89f0-c22163568979","Type":"ContainerDied","Data":"874488b95670c3078fd206e79eb89fb510f58632d62b6a81e4c02f9372f85146"}
Oct 30 19:46:29.662372 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:29.662299    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-ef9a5a-59fc6bf976-zfzks" event={"ID":"09920b13-483e-47d3-89f0-c22163568979","Type":"ContainerDied","Data":"4f9e9fc15c8a115c6748a53ff2a5dab38eb3240ab074fd016b9f1eee9d3a863d"}
Oct 30 19:46:29.662372 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:29.662331    5907 scope.go:122] "RemoveContainer" containerID="874488b95670c3078fd206e79eb89fb510f58632d62b6a81e4c02f9372f85146"
Oct 30 19:46:29.664053 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:29.664019    5907 generic.go:356] "Generic (PLEG): container finished" podID="3b797e1f-bd98-4740-b15d-6ee4e1029d2a" containerID="d3a10aa6120cd43ac32dcff62999d39a94e839e75b0014873b4612e426663b6b" exitCode=0
Oct 30 19:46:29.664169 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:29.664065    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ef9a5a-796b5cf9dc-kmcg7"
Oct 30 19:46:29.664169 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:29.664068    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ef9a5a-796b5cf9dc-kmcg7" event={"ID":"3b797e1f-bd98-4740-b15d-6ee4e1029d2a","Type":"ContainerDied","Data":"d3a10aa6120cd43ac32dcff62999d39a94e839e75b0014873b4612e426663b6b"}
Oct 30 19:46:29.664169 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:29.664108    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-ef9a5a-796b5cf9dc-kmcg7" event={"ID":"3b797e1f-bd98-4740-b15d-6ee4e1029d2a","Type":"ContainerDied","Data":"5d5e530766e0c63fab7e6fe79da5bdd98b52907f3c32c3254dc0408d152ae067"}
Oct 30 19:46:29.675948 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:29.675928    5907 scope.go:122] "RemoveContainer" containerID="874488b95670c3078fd206e79eb89fb510f58632d62b6a81e4c02f9372f85146"
Oct 30 19:46:29.676227 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:46:29.676207    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"874488b95670c3078fd206e79eb89fb510f58632d62b6a81e4c02f9372f85146\": container with ID starting with 874488b95670c3078fd206e79eb89fb510f58632d62b6a81e4c02f9372f85146 not found: ID does not exist" containerID="874488b95670c3078fd206e79eb89fb510f58632d62b6a81e4c02f9372f85146"
Oct 30 19:46:29.676267 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:29.676237    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"874488b95670c3078fd206e79eb89fb510f58632d62b6a81e4c02f9372f85146"} err="failed to get container status \"874488b95670c3078fd206e79eb89fb510f58632d62b6a81e4c02f9372f85146\": rpc error: code = NotFound desc = could not find container \"874488b95670c3078fd206e79eb89fb510f58632d62b6a81e4c02f9372f85146\": container with ID starting with 874488b95670c3078fd206e79eb89fb510f58632d62b6a81e4c02f9372f85146 not found: ID does not exist"
Oct 30 19:46:29.676267 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:29.676253    5907 scope.go:122] "RemoveContainer" containerID="d3a10aa6120cd43ac32dcff62999d39a94e839e75b0014873b4612e426663b6b"
Oct 30 19:46:29.686464 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:29.686445    5907 scope.go:122] "RemoveContainer" containerID="d3a10aa6120cd43ac32dcff62999d39a94e839e75b0014873b4612e426663b6b"
Oct 30 19:46:29.686738 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:46:29.686718    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3a10aa6120cd43ac32dcff62999d39a94e839e75b0014873b4612e426663b6b\": container with ID starting with d3a10aa6120cd43ac32dcff62999d39a94e839e75b0014873b4612e426663b6b not found: ID does not exist" containerID="d3a10aa6120cd43ac32dcff62999d39a94e839e75b0014873b4612e426663b6b"
Oct 30 19:46:29.686781 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:29.686748    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3a10aa6120cd43ac32dcff62999d39a94e839e75b0014873b4612e426663b6b"} err="failed to get container status \"d3a10aa6120cd43ac32dcff62999d39a94e839e75b0014873b4612e426663b6b\": rpc error: code = NotFound desc = could not find container \"d3a10aa6120cd43ac32dcff62999d39a94e839e75b0014873b4612e426663b6b\": container with ID starting with d3a10aa6120cd43ac32dcff62999d39a94e839e75b0014873b4612e426663b6b not found: ID does not exist"
Oct 30 19:46:29.730418 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:29.730388    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-ef9a5a-796b5cf9dc-kmcg7"]
Oct 30 19:46:29.744965 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:29.744936    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-ef9a5a-796b5cf9dc-kmcg7"]
Oct 30 19:46:29.787349 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:29.787324    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-ef9a5a-59fc6bf976-zfzks"]
Oct 30 19:46:29.795497 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:29.795473    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-ef9a5a-59fc6bf976-zfzks"]
Oct 30 19:46:30.289034 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:30.289003    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="09920b13-483e-47d3-89f0-c22163568979" path="/var/lib/kubelet/pods/09920b13-483e-47d3-89f0-c22163568979/volumes"
Oct 30 19:46:30.289439 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:30.289397    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3b797e1f-bd98-4740-b15d-6ee4e1029d2a" path="/var/lib/kubelet/pods/3b797e1f-bd98-4740-b15d-6ee4e1029d2a/volumes"
Oct 30 19:46:31.233416 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.233380    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"]
Oct 30 19:46:31.233654 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.233628    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh" podUID="b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf" containerName="istio-proxy" containerID="cri-o://5f09ff5c14f414d2bbf80f8daacea879457e1604afa0ab2eb0a3bf37de89d037" gracePeriod=30
Oct 30 19:46:31.761570 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.761532    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9de441-4zkxr"]
Oct 30 19:46:31.762197 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.762182    5907 cpu_manager.go:414] "containerMap: removing container" podUID="81eea74f-1fd9-41b0-8ecf-0501402c6a66" containerName="istio-proxy"
Oct 30 19:46:31.762249 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.762199    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="81eea74f-1fd9-41b0-8ecf-0501402c6a66" containerName="istio-proxy"
Oct 30 19:46:31.762249 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.762212    5907 cpu_manager.go:414] "containerMap: removing container" podUID="6c3bc483-1c8c-43d7-a336-ee6e04eda24b" containerName="payload-pre-processing"
Oct 30 19:46:31.762249 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.762217    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6c3bc483-1c8c-43d7-a336-ee6e04eda24b" containerName="payload-pre-processing"
Oct 30 19:46:31.762249 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.762236    5907 cpu_manager.go:414] "containerMap: removing container" podUID="9dd73dbe-af1c-4d56-82e3-228644efda7a" containerName="revoke-keys"
Oct 30 19:46:31.762249 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.762241    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9dd73dbe-af1c-4d56-82e3-228644efda7a" containerName="revoke-keys"
Oct 30 19:46:31.762422 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.762253    5907 cpu_manager.go:414] "containerMap: removing container" podUID="5dfc7248-d1a7-4093-a2b8-eaf603d9a660" containerName="payload-processing"
Oct 30 19:46:31.762422 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.762258    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5dfc7248-d1a7-4093-a2b8-eaf603d9a660" containerName="payload-processing"
Oct 30 19:46:31.762422 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.762265    5907 cpu_manager.go:414] "containerMap: removing container" podUID="57838872-c482-41ec-bda0-4a517fb81dcd" containerName="istio-proxy"
Oct 30 19:46:31.762422 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.762269    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="57838872-c482-41ec-bda0-4a517fb81dcd" containerName="istio-proxy"
Oct 30 19:46:31.762422 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.762289    5907 cpu_manager.go:414] "containerMap: removing container" podUID="57838872-c482-41ec-bda0-4a517fb81dcd" containerName="istio-proxy"
Oct 30 19:46:31.762422 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.762294    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="57838872-c482-41ec-bda0-4a517fb81dcd" containerName="istio-proxy"
Oct 30 19:46:31.762422 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.762309    5907 cpu_manager.go:414] "containerMap: removing container" podUID="282f4fd8-512c-41b9-9989-0b84f2de1ce5" containerName="istio-proxy"
Oct 30 19:46:31.762422 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.762313    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="282f4fd8-512c-41b9-9989-0b84f2de1ce5" containerName="istio-proxy"
Oct 30 19:46:31.762422 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.762329    5907 cpu_manager.go:414] "containerMap: removing container" podUID="4a53e23a-fa9d-43f9-a651-baa25612b463" containerName="revoke-keys"
Oct 30 19:46:31.762422 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.762333    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4a53e23a-fa9d-43f9-a651-baa25612b463" containerName="revoke-keys"
Oct 30 19:46:31.762422 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.762341    5907 cpu_manager.go:414] "containerMap: removing container" podUID="282f4fd8-512c-41b9-9989-0b84f2de1ce5" containerName="istio-proxy"
Oct 30 19:46:31.762422 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.762345    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="282f4fd8-512c-41b9-9989-0b84f2de1ce5" containerName="istio-proxy"
Oct 30 19:46:31.762422 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.762353    5907 cpu_manager.go:414] "containerMap: removing container" podUID="3b797e1f-bd98-4740-b15d-6ee4e1029d2a" containerName="payload-pre-processing"
Oct 30 19:46:31.762422 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.762357    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3b797e1f-bd98-4740-b15d-6ee4e1029d2a" containerName="payload-pre-processing"
Oct 30 19:46:31.762422 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.762364    5907 cpu_manager.go:414] "containerMap: removing container" podUID="09920b13-483e-47d3-89f0-c22163568979" containerName="payload-processing"
Oct 30 19:46:31.762422 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.762369    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="09920b13-483e-47d3-89f0-c22163568979" containerName="payload-processing"
Oct 30 19:46:31.762422 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.762376    5907 cpu_manager.go:414] "containerMap: removing container" podUID="b9d6296e-59ef-4853-8470-d851752d8694" containerName="revoke-keys"
Oct 30 19:46:31.762422 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.762380    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b9d6296e-59ef-4853-8470-d851752d8694" containerName="revoke-keys"
Oct 30 19:46:31.762993 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.762463    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="57838872-c482-41ec-bda0-4a517fb81dcd" containerName="istio-proxy"
Oct 30 19:46:31.762993 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.762474    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="282f4fd8-512c-41b9-9989-0b84f2de1ce5" containerName="istio-proxy"
Oct 30 19:46:31.762993 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.762483    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="4a53e23a-fa9d-43f9-a651-baa25612b463" containerName="revoke-keys"
Oct 30 19:46:31.762993 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.762490    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="282f4fd8-512c-41b9-9989-0b84f2de1ce5" containerName="istio-proxy"
Oct 30 19:46:31.762993 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.762495    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="3b797e1f-bd98-4740-b15d-6ee4e1029d2a" containerName="payload-pre-processing"
Oct 30 19:46:31.762993 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.762501    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="09920b13-483e-47d3-89f0-c22163568979" containerName="payload-processing"
Oct 30 19:46:31.762993 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.762507    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="b9d6296e-59ef-4853-8470-d851752d8694" containerName="revoke-keys"
Oct 30 19:46:31.762993 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.762512    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="81eea74f-1fd9-41b0-8ecf-0501402c6a66" containerName="istio-proxy"
Oct 30 19:46:31.762993 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.762518    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="6c3bc483-1c8c-43d7-a336-ee6e04eda24b" containerName="payload-pre-processing"
Oct 30 19:46:31.762993 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.762525    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="9dd73dbe-af1c-4d56-82e3-228644efda7a" containerName="revoke-keys"
Oct 30 19:46:31.762993 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.762530    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="5dfc7248-d1a7-4093-a2b8-eaf603d9a660" containerName="payload-processing"
Oct 30 19:46:31.767898 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.767879    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9de441-4zkxr"
Oct 30 19:46:31.791196 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.791174    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:46:31.791504 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.791485    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:46:31.799460 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.799433    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9de441-4zkxr"]
Oct 30 19:46:31.900223 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.900187    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4179be3f-8227-4cab-aa4a-62825fa5fe03-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-api-b-9de441-4zkxr\" (UID: \"4179be3f-8227-4cab-aa4a-62825fa5fe03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9de441-4zkxr"
Oct 30 19:46:31.900411 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.900290    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4179be3f-8227-4cab-aa4a-62825fa5fe03-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-api-b-9de441-4zkxr\" (UID: \"4179be3f-8227-4cab-aa4a-62825fa5fe03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9de441-4zkxr"
Oct 30 19:46:31.900411 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:31.900354    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw9bh\" (UniqueName: \"kubernetes.io/projected/4179be3f-8227-4cab-aa4a-62825fa5fe03-kube-api-access-nw9bh\") pod \"maas-api-revoke-keys-e2e-api-b-9de441-4zkxr\" (UID: \"4179be3f-8227-4cab-aa4a-62825fa5fe03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9de441-4zkxr"
Oct 30 19:46:32.001258 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:32.001224    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4179be3f-8227-4cab-aa4a-62825fa5fe03-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-api-b-9de441-4zkxr\" (UID: \"4179be3f-8227-4cab-aa4a-62825fa5fe03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9de441-4zkxr"
Oct 30 19:46:32.001429 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:32.001305    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4179be3f-8227-4cab-aa4a-62825fa5fe03-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-api-b-9de441-4zkxr\" (UID: \"4179be3f-8227-4cab-aa4a-62825fa5fe03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9de441-4zkxr"
Oct 30 19:46:32.001429 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:32.001340    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw9bh\" (UniqueName: \"kubernetes.io/projected/4179be3f-8227-4cab-aa4a-62825fa5fe03-kube-api-access-nw9bh\") pod \"maas-api-revoke-keys-e2e-api-b-9de441-4zkxr\" (UID: \"4179be3f-8227-4cab-aa4a-62825fa5fe03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9de441-4zkxr"
Oct 30 19:46:32.002014 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:32.001994    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4179be3f-8227-4cab-aa4a-62825fa5fe03-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-api-b-9de441-4zkxr\" (UID: \"4179be3f-8227-4cab-aa4a-62825fa5fe03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9de441-4zkxr"
Oct 30 19:46:32.003805 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:32.003780    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4179be3f-8227-4cab-aa4a-62825fa5fe03-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-api-b-9de441-4zkxr\" (UID: \"4179be3f-8227-4cab-aa4a-62825fa5fe03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9de441-4zkxr"
Oct 30 19:46:32.023988 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:32.023931    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw9bh\" (UniqueName: \"kubernetes.io/projected/4179be3f-8227-4cab-aa4a-62825fa5fe03-kube-api-access-nw9bh\") pod \"maas-api-revoke-keys-e2e-api-b-9de441-4zkxr\" (UID: \"4179be3f-8227-4cab-aa4a-62825fa5fe03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9de441-4zkxr"
Oct 30 19:46:32.081769 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:32.081735    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9de441-4zkxr"
Oct 30 19:46:32.418330 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:32.418299    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9de441-4zkxr"]
Oct 30 19:46:32.419659 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:46:32.419613    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4179be3f_8227_4cab_aa4a_62825fa5fe03.slice/crio-00c3d974de0bd6016db14428d9b803f56d1ab3e00abe65d38f8cac81091cb8fa WatchSource:0}: Error finding container 00c3d974de0bd6016db14428d9b803f56d1ab3e00abe65d38f8cac81091cb8fa: Status 404 returned error can't find the container with id 00c3d974de0bd6016db14428d9b803f56d1ab3e00abe65d38f8cac81091cb8fa
Oct 30 19:46:32.704738 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:32.704649    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9de441-4zkxr" event={"ID":"4179be3f-8227-4cab-aa4a-62825fa5fe03","Type":"ContainerStarted","Data":"00c3d974de0bd6016db14428d9b803f56d1ab3e00abe65d38f8cac81091cb8fa"}
Oct 30 19:46:32.938943 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:32.938896    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-4b4c34-5bb88fdc94-s44w4"]
Oct 30 19:46:32.966766 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:32.966688    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-4b4c34-556f66b87f-d864z"]
Oct 30 19:46:33.692043 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:33.692008    5907 scope.go:122] "RemoveContainer" containerID="fd88a915d82e01878f1b18fc95a7a462441d0f7674bc43dbbf1cc7940abd5b61"
Oct 30 19:46:33.714756 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:33.714725    5907 generic.go:356] "Generic (PLEG): container finished" podID="4179be3f-8227-4cab-aa4a-62825fa5fe03" containerID="b0c8803df220726b8f52408e2436d652f70252dda2740e4e7c73c970ae1bfc57" exitCode=0
Oct 30 19:46:33.714941 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:33.714804    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9de441-4zkxr" event={"ID":"4179be3f-8227-4cab-aa4a-62825fa5fe03","Type":"ContainerDied","Data":"b0c8803df220726b8f52408e2436d652f70252dda2740e4e7c73c970ae1bfc57"}
Oct 30 19:46:34.810109 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:34.806715    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:46:35.071202 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:35.071178    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9de441-4zkxr"
Oct 30 19:46:35.113667 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:35.113630    5907 patch_prober.go:29] interesting pod/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:46:35.113854 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:35.113689    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh" podUID="b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:46:35.177721 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:35.177692    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4179be3f-8227-4cab-aa4a-62825fa5fe03-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4179be3f-8227-4cab-aa4a-62825fa5fe03-maas-api-openshift-ca\") pod \"4179be3f-8227-4cab-aa4a-62825fa5fe03\" (UID: \"4179be3f-8227-4cab-aa4a-62825fa5fe03\") "
Oct 30 19:46:35.177959 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:35.177816    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4179be3f-8227-4cab-aa4a-62825fa5fe03-kube-api-access-nw9bh\" (UniqueName: \"kubernetes.io/projected/4179be3f-8227-4cab-aa4a-62825fa5fe03-kube-api-access-nw9bh\") pod \"4179be3f-8227-4cab-aa4a-62825fa5fe03\" (UID: \"4179be3f-8227-4cab-aa4a-62825fa5fe03\") "
Oct 30 19:46:35.177959 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:35.177853    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4179be3f-8227-4cab-aa4a-62825fa5fe03-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4179be3f-8227-4cab-aa4a-62825fa5fe03-maas-api-xks-ca\") pod \"4179be3f-8227-4cab-aa4a-62825fa5fe03\" (UID: \"4179be3f-8227-4cab-aa4a-62825fa5fe03\") "
Oct 30 19:46:35.178173 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:35.178149    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4179be3f-8227-4cab-aa4a-62825fa5fe03-maas-api-openshift-ca" pod "4179be3f-8227-4cab-aa4a-62825fa5fe03" (UID: "4179be3f-8227-4cab-aa4a-62825fa5fe03"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:35.178424 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:35.178407    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4179be3f-8227-4cab-aa4a-62825fa5fe03-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:35.180107 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:35.180078    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4179be3f-8227-4cab-aa4a-62825fa5fe03-kube-api-access-nw9bh" pod "4179be3f-8227-4cab-aa4a-62825fa5fe03" (UID: "4179be3f-8227-4cab-aa4a-62825fa5fe03"). InnerVolumeSpecName "kube-api-access-nw9bh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:35.180203 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:35.180080    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4179be3f-8227-4cab-aa4a-62825fa5fe03-maas-api-xks-ca" pod "4179be3f-8227-4cab-aa4a-62825fa5fe03" (UID: "4179be3f-8227-4cab-aa4a-62825fa5fe03"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:46:35.280180 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:35.280141    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nw9bh\" (UniqueName: \"kubernetes.io/projected/4179be3f-8227-4cab-aa4a-62825fa5fe03-kube-api-access-nw9bh\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:35.280180 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:35.280171    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4179be3f-8227-4cab-aa4a-62825fa5fe03-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:35.730511 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:35.730478    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9de441-4zkxr"
Oct 30 19:46:35.730727 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:35.730504    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9de441-4zkxr" event={"ID":"4179be3f-8227-4cab-aa4a-62825fa5fe03","Type":"ContainerDied","Data":"00c3d974de0bd6016db14428d9b803f56d1ab3e00abe65d38f8cac81091cb8fa"}
Oct 30 19:46:35.730727 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:35.730559    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="00c3d974de0bd6016db14428d9b803f56d1ab3e00abe65d38f8cac81091cb8fa"
Oct 30 19:46:36.505506 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.505440    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:46:36.619108 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.619072    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-kube-api-access-ddkrs\" (UniqueName: \"kubernetes.io/projected/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-kube-api-access-ddkrs\") pod \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") "
Oct 30 19:46:36.619310 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.619168    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-workload-socket\") pod \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") "
Oct 30 19:46:36.619310 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.619211    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-credential-socket\") pod \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") "
Oct 30 19:46:36.619310 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.619250    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-workload-certs\") pod \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") "
Oct 30 19:46:36.619501 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.619387    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-istio-token\" (UniqueName: \"kubernetes.io/projected/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-istio-token\") pod \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") "
Oct 30 19:46:36.619501 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.619397    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-workload-socket" pod "b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf" (UID: "b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:36.619501 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.619420    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-istio-podinfo\") pod \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") "
Oct 30 19:46:36.619501 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.619463    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-istio-data\") pod \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") "
Oct 30 19:46:36.619728 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.619517    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-istiod-ca-cert\") pod \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") "
Oct 30 19:46:36.619728 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.619560    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-istio-envoy\") pod \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\" (UID: \"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf\") "
Oct 30 19:46:36.619728 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.619562    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-workload-certs" pod "b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf" (UID: "b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:36.619910 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.619729    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-credential-socket" pod "b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf" (UID: "b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:36.620168 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.620141    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-istio-data" pod "b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf" (UID: "b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:36.620259 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.620246    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:36.620319 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.620269    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:36.620319 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.620285    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:36.620319 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.620299    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:36.620319 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.620268    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-istiod-ca-cert" pod "b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf" (UID: "b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:36.622571 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.622541    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-kube-api-access-ddkrs" pod "b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf" (UID: "b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf"). InnerVolumeSpecName "kube-api-access-ddkrs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:36.622780 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.622754    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-istio-podinfo" pod "b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf" (UID: "b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:46:36.622955 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.622931    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-istio-envoy" pod "b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf" (UID: "b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:36.623218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.623187    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-istio-token" pod "b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf" (UID: "b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:36.724884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.722257    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:36.724884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.722302    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:36.724884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.722319    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:36.724884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.722333    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:36.724884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.722347    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddkrs\" (UniqueName: \"kubernetes.io/projected/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf-kube-api-access-ddkrs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:36.768603 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.763164    5907 generic.go:356] "Generic (PLEG): container finished" podID="b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf" containerID="5f09ff5c14f414d2bbf80f8daacea879457e1604afa0ab2eb0a3bf37de89d037" exitCode=0
Oct 30 19:46:36.768603 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.763312    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh" event={"ID":"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf","Type":"ContainerDied","Data":"5f09ff5c14f414d2bbf80f8daacea879457e1604afa0ab2eb0a3bf37de89d037"}
Oct 30 19:46:36.768603 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.763365    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh" event={"ID":"b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf","Type":"ContainerDied","Data":"368654e63ab0482f8098e8620b1130eb9d6bda7e6105070fe74607fb77c14a8c"}
Oct 30 19:46:36.768603 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.763399    5907 scope.go:122] "RemoveContainer" containerID="5f09ff5c14f414d2bbf80f8daacea879457e1604afa0ab2eb0a3bf37de89d037"
Oct 30 19:46:36.768603 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.763578    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"
Oct 30 19:46:36.788987 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.785555    5907 scope.go:122] "RemoveContainer" containerID="28b30d77ae7ed333981688e96cd6f906eea4c288ff68b8efb0764bbfe4fd7b65"
Oct 30 19:46:36.796911 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.796873    5907 scope.go:122] "RemoveContainer" containerID="5f09ff5c14f414d2bbf80f8daacea879457e1604afa0ab2eb0a3bf37de89d037"
Oct 30 19:46:36.797158 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:46:36.797139    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f09ff5c14f414d2bbf80f8daacea879457e1604afa0ab2eb0a3bf37de89d037\": container with ID starting with 5f09ff5c14f414d2bbf80f8daacea879457e1604afa0ab2eb0a3bf37de89d037 not found: ID does not exist" containerID="5f09ff5c14f414d2bbf80f8daacea879457e1604afa0ab2eb0a3bf37de89d037"
Oct 30 19:46:36.797215 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.797166    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f09ff5c14f414d2bbf80f8daacea879457e1604afa0ab2eb0a3bf37de89d037"} err="failed to get container status \"5f09ff5c14f414d2bbf80f8daacea879457e1604afa0ab2eb0a3bf37de89d037\": rpc error: code = NotFound desc = could not find container \"5f09ff5c14f414d2bbf80f8daacea879457e1604afa0ab2eb0a3bf37de89d037\": container with ID starting with 5f09ff5c14f414d2bbf80f8daacea879457e1604afa0ab2eb0a3bf37de89d037 not found: ID does not exist"
Oct 30 19:46:36.797215 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.797190    5907 scope.go:122] "RemoveContainer" containerID="28b30d77ae7ed333981688e96cd6f906eea4c288ff68b8efb0764bbfe4fd7b65"
Oct 30 19:46:36.797533 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:46:36.797506    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28b30d77ae7ed333981688e96cd6f906eea4c288ff68b8efb0764bbfe4fd7b65\": container with ID starting with 28b30d77ae7ed333981688e96cd6f906eea4c288ff68b8efb0764bbfe4fd7b65 not found: ID does not exist" containerID="28b30d77ae7ed333981688e96cd6f906eea4c288ff68b8efb0764bbfe4fd7b65"
Oct 30 19:46:36.797657 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.797536    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28b30d77ae7ed333981688e96cd6f906eea4c288ff68b8efb0764bbfe4fd7b65"} err="failed to get container status \"28b30d77ae7ed333981688e96cd6f906eea4c288ff68b8efb0764bbfe4fd7b65\": rpc error: code = NotFound desc = could not find container \"28b30d77ae7ed333981688e96cd6f906eea4c288ff68b8efb0764bbfe4fd7b65\": container with ID starting with 28b30d77ae7ed333981688e96cd6f906eea4c288ff68b8efb0764bbfe4fd7b65 not found: ID does not exist"
Oct 30 19:46:36.826935 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.826906    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"]
Oct 30 19:46:36.835538 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:36.835512    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-3add95-openshift-default-5d656856f-rt2nh"]
Oct 30 19:46:37.249977 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:37.245725    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-fe4ec3-jrzc5"]
Oct 30 19:46:37.249977 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:37.246800    5907 cpu_manager.go:414] "containerMap: removing container" podUID="b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf" containerName="istio-proxy"
Oct 30 19:46:37.249977 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:37.246820    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf" containerName="istio-proxy"
Oct 30 19:46:37.249977 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:37.246850    5907 cpu_manager.go:414] "containerMap: removing container" podUID="4179be3f-8227-4cab-aa4a-62825fa5fe03" containerName="revoke-keys"
Oct 30 19:46:37.249977 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:37.246877    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4179be3f-8227-4cab-aa4a-62825fa5fe03" containerName="revoke-keys"
Oct 30 19:46:37.249977 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:37.246892    5907 cpu_manager.go:414] "containerMap: removing container" podUID="b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf" containerName="istio-proxy"
Oct 30 19:46:37.249977 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:37.246898    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf" containerName="istio-proxy"
Oct 30 19:46:37.249977 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:37.247128    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="57838872-c482-41ec-bda0-4a517fb81dcd" containerName="istio-proxy"
Oct 30 19:46:37.249977 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:37.247144    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf" containerName="istio-proxy"
Oct 30 19:46:37.249977 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:37.247156    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="4179be3f-8227-4cab-aa4a-62825fa5fe03" containerName="revoke-keys"
Oct 30 19:46:37.275042 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:37.274233    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-fe4ec3-jrzc5"
Oct 30 19:46:37.286890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:37.283913    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-fe4ec3-jrzc5"]
Oct 30 19:46:37.363885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:37.321350    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:46:37.363885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:37.321433    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:46:37.363885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:37.352769    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qb87p\" (UniqueName: \"kubernetes.io/projected/0ddee3f4-7acc-44ed-affc-d0096c44b37a-kube-api-access-qb87p\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-fe4ec3-jrzc5\" (UID: \"0ddee3f4-7acc-44ed-affc-d0096c44b37a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-fe4ec3-jrzc5"
Oct 30 19:46:37.363885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:37.352932    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0ddee3f4-7acc-44ed-affc-d0096c44b37a-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-fe4ec3-jrzc5\" (UID: \"0ddee3f4-7acc-44ed-affc-d0096c44b37a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-fe4ec3-jrzc5"
Oct 30 19:46:37.363885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:37.353018    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0ddee3f4-7acc-44ed-affc-d0096c44b37a-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-fe4ec3-jrzc5\" (UID: \"0ddee3f4-7acc-44ed-affc-d0096c44b37a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-fe4ec3-jrzc5"
Oct 30 19:46:37.469891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:37.469668    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0ddee3f4-7acc-44ed-affc-d0096c44b37a-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-fe4ec3-jrzc5\" (UID: \"0ddee3f4-7acc-44ed-affc-d0096c44b37a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-fe4ec3-jrzc5"
Oct 30 19:46:37.470127 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:37.470035    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qb87p\" (UniqueName: \"kubernetes.io/projected/0ddee3f4-7acc-44ed-affc-d0096c44b37a-kube-api-access-qb87p\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-fe4ec3-jrzc5\" (UID: \"0ddee3f4-7acc-44ed-affc-d0096c44b37a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-fe4ec3-jrzc5"
Oct 30 19:46:37.470285 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:37.470226    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0ddee3f4-7acc-44ed-affc-d0096c44b37a-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-fe4ec3-jrzc5\" (UID: \"0ddee3f4-7acc-44ed-affc-d0096c44b37a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-fe4ec3-jrzc5"
Oct 30 19:46:37.477153 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:37.471706    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0ddee3f4-7acc-44ed-affc-d0096c44b37a-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-fe4ec3-jrzc5\" (UID: \"0ddee3f4-7acc-44ed-affc-d0096c44b37a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-fe4ec3-jrzc5"
Oct 30 19:46:37.479347 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:37.478220    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0ddee3f4-7acc-44ed-affc-d0096c44b37a-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-fe4ec3-jrzc5\" (UID: \"0ddee3f4-7acc-44ed-affc-d0096c44b37a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-fe4ec3-jrzc5"
Oct 30 19:46:37.494100 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:37.494070    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qb87p\" (UniqueName: \"kubernetes.io/projected/0ddee3f4-7acc-44ed-affc-d0096c44b37a-kube-api-access-qb87p\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-fe4ec3-jrzc5\" (UID: \"0ddee3f4-7acc-44ed-affc-d0096c44b37a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-fe4ec3-jrzc5"
Oct 30 19:46:37.641153 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:37.641113    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-fe4ec3-jrzc5"
Oct 30 19:46:37.940367 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:37.939133    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-4b4c34-5bb88fdc94-s44w4" podUID="2dee0a84-130e-413d-8294-a86bf9c82c92" containerName="payload-processing" containerID="cri-o://a312f454db6b8f05a75ca06ed4c412547765da1bc3784654cb3ecf916aada2b5" gracePeriod=25
Oct 30 19:46:37.967410 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:37.967341    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b4c34-556f66b87f-d864z" podUID="b9fbde13-05e2-47a9-93bc-65cbd47efa6a" containerName="payload-pre-processing" containerID="cri-o://e3f6c6e636856fe207c94d3ce951e8abfe0e683aecc943afd6d2dcbf073e792d" gracePeriod=25
Oct 30 19:46:37.988782 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:37.988740    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-fe4ec3-jrzc5"]
Oct 30 19:46:38.288935 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:38.288833    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf" path="/var/lib/kubelet/pods/b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf/volumes"
Oct 30 19:46:38.787451 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:38.787404    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-fe4ec3-jrzc5" event={"ID":"0ddee3f4-7acc-44ed-affc-d0096c44b37a","Type":"ContainerStarted","Data":"40df6df7a820baea9118e3e42586258d0c691515a33b827ede5fade8425d8a09"}
Oct 30 19:46:38.787938 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:38.787493    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-fe4ec3-jrzc5" event={"ID":"0ddee3f4-7acc-44ed-affc-d0096c44b37a","Type":"ContainerStarted","Data":"41d136d4b0edf4532f9ca2d30b6273a3e59a71e4399a0b34099131599781a1e1"}
Oct 30 19:46:38.831191 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:38.831130    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-fe4ec3-jrzc5" podStartSLOduration=1.831111399 podStartE2EDuration="1.831111399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:46:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:46:38.829476132 +0000 UTC m=+1510.672408072" watchObservedRunningTime="2026-10-30 19:46:38.831111399 +0000 UTC m=+1510.674043383"
Oct 30 19:46:39.795651 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:39.795612    5907 generic.go:356] "Generic (PLEG): container finished" podID="0ddee3f4-7acc-44ed-affc-d0096c44b37a" containerID="40df6df7a820baea9118e3e42586258d0c691515a33b827ede5fade8425d8a09" exitCode=0
Oct 30 19:46:39.796216 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:39.795665    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-fe4ec3-jrzc5" event={"ID":"0ddee3f4-7acc-44ed-affc-d0096c44b37a","Type":"ContainerDied","Data":"40df6df7a820baea9118e3e42586258d0c691515a33b827ede5fade8425d8a09"}
Oct 30 19:46:40.090940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:40.090472    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-4b4c34-5bb88fdc94-s44w4" podUID="2dee0a84-130e-413d-8294-a86bf9c82c92" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:46:40.110888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:40.109708    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b4c34-556f66b87f-d864z" podUID="b9fbde13-05e2-47a9-93bc-65cbd47efa6a" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:46:40.515810 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:40.515776    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ef9a5a-wxc7j"]
Oct 30 19:46:40.528483 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:40.528449    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-ef9a5a-wxc7j"]
Oct 30 19:46:40.970647 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:40.970624    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-fe4ec3-jrzc5"
Oct 30 19:46:41.160124 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:41.160093    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0ddee3f4-7acc-44ed-affc-d0096c44b37a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0ddee3f4-7acc-44ed-affc-d0096c44b37a-maas-api-openshift-ca\") pod \"0ddee3f4-7acc-44ed-affc-d0096c44b37a\" (UID: \"0ddee3f4-7acc-44ed-affc-d0096c44b37a\") "
Oct 30 19:46:41.160339 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:41.160220    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0ddee3f4-7acc-44ed-affc-d0096c44b37a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0ddee3f4-7acc-44ed-affc-d0096c44b37a-maas-api-xks-ca\") pod \"0ddee3f4-7acc-44ed-affc-d0096c44b37a\" (UID: \"0ddee3f4-7acc-44ed-affc-d0096c44b37a\") "
Oct 30 19:46:41.160431 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:41.160384    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ddee3f4-7acc-44ed-affc-d0096c44b37a-kube-api-access-qb87p\" (UniqueName: \"kubernetes.io/projected/0ddee3f4-7acc-44ed-affc-d0096c44b37a-kube-api-access-qb87p\") pod \"0ddee3f4-7acc-44ed-affc-d0096c44b37a\" (UID: \"0ddee3f4-7acc-44ed-affc-d0096c44b37a\") "
Oct 30 19:46:41.160525 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:41.160498    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ddee3f4-7acc-44ed-affc-d0096c44b37a-maas-api-openshift-ca" pod "0ddee3f4-7acc-44ed-affc-d0096c44b37a" (UID: "0ddee3f4-7acc-44ed-affc-d0096c44b37a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:41.160817 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:41.160798    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0ddee3f4-7acc-44ed-affc-d0096c44b37a-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:41.162509 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:41.162485    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ddee3f4-7acc-44ed-affc-d0096c44b37a-maas-api-xks-ca" pod "0ddee3f4-7acc-44ed-affc-d0096c44b37a" (UID: "0ddee3f4-7acc-44ed-affc-d0096c44b37a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:46:41.162642 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:41.162621    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ddee3f4-7acc-44ed-affc-d0096c44b37a-kube-api-access-qb87p" pod "0ddee3f4-7acc-44ed-affc-d0096c44b37a" (UID: "0ddee3f4-7acc-44ed-affc-d0096c44b37a"). InnerVolumeSpecName "kube-api-access-qb87p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:41.262722 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:41.262681    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0ddee3f4-7acc-44ed-affc-d0096c44b37a-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:41.262722 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:41.262722    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qb87p\" (UniqueName: \"kubernetes.io/projected/0ddee3f4-7acc-44ed-affc-d0096c44b37a-kube-api-access-qb87p\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:41.813955 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:41.813923    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-fe4ec3-jrzc5"
Oct 30 19:46:41.814163 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:41.813922    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-fe4ec3-jrzc5" event={"ID":"0ddee3f4-7acc-44ed-affc-d0096c44b37a","Type":"ContainerDied","Data":"41d136d4b0edf4532f9ca2d30b6273a3e59a71e4399a0b34099131599781a1e1"}
Oct 30 19:46:41.814163 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:41.814053    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="41d136d4b0edf4532f9ca2d30b6273a3e59a71e4399a0b34099131599781a1e1"
Oct 30 19:46:41.962919 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:41.962879    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main" containerID="cri-o://02ae6121b780eea9dbffa8ba7a0ce00e619740331de6c77558feb4f596bf90f5" gracePeriod=45
Oct 30 19:46:42.143119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.143055    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6" podUID="c3f73773-6c2f-47e4-a5a4-c57916f64ac9" containerName="main" containerID="cri-o://73be7a140f7c340a40c5680df706a9beff4c8be230f47f0ee20ffb1ec73d3071" gracePeriod=45
Oct 30 19:46:42.173374 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.173345    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9de441-ffb9d7698-7bgvb"]
Oct 30 19:46:42.206604 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.206573    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9de441-67cfb6c4c5-vgswm"]
Oct 30 19:46:42.224466 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.224434    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6" podUID="c3f73773-6c2f-47e4-a5a4-c57916f64ac9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.176:8000/ready\": dial tcp 10.134.0.176:8000: connect: connection refused"
Oct 30 19:46:42.294107 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.294075    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="810e8003-26f9-422c-a7e8-9cb10c76ce45" path="/var/lib/kubelet/pods/810e8003-26f9-422c-a7e8-9cb10c76ce45/volumes"
Oct 30 19:46:42.305905 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.298929    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:46:42.375782 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.375701    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f127fc10-e30f-405d-b5cf-be0d45cdee6e-tls-certs\" (UniqueName: \"kubernetes.io/secret/f127fc10-e30f-405d-b5cf-be0d45cdee6e-tls-certs\") pod \"f127fc10-e30f-405d-b5cf-be0d45cdee6e\" (UID: \"f127fc10-e30f-405d-b5cf-be0d45cdee6e\") "
Oct 30 19:46:42.376009 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.375807    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f127fc10-e30f-405d-b5cf-be0d45cdee6e-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f127fc10-e30f-405d-b5cf-be0d45cdee6e-tmp-dir\") pod \"f127fc10-e30f-405d-b5cf-be0d45cdee6e\" (UID: \"f127fc10-e30f-405d-b5cf-be0d45cdee6e\") "
Oct 30 19:46:42.377946 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.376232    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f127fc10-e30f-405d-b5cf-be0d45cdee6e-tmp-dir" pod "f127fc10-e30f-405d-b5cf-be0d45cdee6e" (UID: "f127fc10-e30f-405d-b5cf-be0d45cdee6e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:42.377946 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.376346    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f127fc10-e30f-405d-b5cf-be0d45cdee6e-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f127fc10-e30f-405d-b5cf-be0d45cdee6e-dshm\") pod \"f127fc10-e30f-405d-b5cf-be0d45cdee6e\" (UID: \"f127fc10-e30f-405d-b5cf-be0d45cdee6e\") "
Oct 30 19:46:42.377946 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.377346    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f127fc10-e30f-405d-b5cf-be0d45cdee6e-kube-api-access-jvlj2\" (UniqueName: \"kubernetes.io/projected/f127fc10-e30f-405d-b5cf-be0d45cdee6e-kube-api-access-jvlj2\") pod \"f127fc10-e30f-405d-b5cf-be0d45cdee6e\" (UID: \"f127fc10-e30f-405d-b5cf-be0d45cdee6e\") "
Oct 30 19:46:42.377946 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.377476    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f127fc10-e30f-405d-b5cf-be0d45cdee6e-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f127fc10-e30f-405d-b5cf-be0d45cdee6e-model-cache\") pod \"f127fc10-e30f-405d-b5cf-be0d45cdee6e\" (UID: \"f127fc10-e30f-405d-b5cf-be0d45cdee6e\") "
Oct 30 19:46:42.381249 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.378293    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f127fc10-e30f-405d-b5cf-be0d45cdee6e-model-cache" pod "f127fc10-e30f-405d-b5cf-be0d45cdee6e" (UID: "f127fc10-e30f-405d-b5cf-be0d45cdee6e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:42.381249 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.378383    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f127fc10-e30f-405d-b5cf-be0d45cdee6e-home\" (UniqueName: \"kubernetes.io/empty-dir/f127fc10-e30f-405d-b5cf-be0d45cdee6e-home\") pod \"f127fc10-e30f-405d-b5cf-be0d45cdee6e\" (UID: \"f127fc10-e30f-405d-b5cf-be0d45cdee6e\") "
Oct 30 19:46:42.381249 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.378641    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f127fc10-e30f-405d-b5cf-be0d45cdee6e-home" pod "f127fc10-e30f-405d-b5cf-be0d45cdee6e" (UID: "f127fc10-e30f-405d-b5cf-be0d45cdee6e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:42.381249 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.379200    5907 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f127fc10-e30f-405d-b5cf-be0d45cdee6e-home\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:42.381249 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.379219    5907 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f127fc10-e30f-405d-b5cf-be0d45cdee6e-tmp-dir\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:42.381249 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.379231    5907 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f127fc10-e30f-405d-b5cf-be0d45cdee6e-model-cache\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:42.381249 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.379800    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f127fc10-e30f-405d-b5cf-be0d45cdee6e-tls-certs" pod "f127fc10-e30f-405d-b5cf-be0d45cdee6e" (UID: "f127fc10-e30f-405d-b5cf-be0d45cdee6e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:46:42.381249 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.380981    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f127fc10-e30f-405d-b5cf-be0d45cdee6e-dshm" pod "f127fc10-e30f-405d-b5cf-be0d45cdee6e" (UID: "f127fc10-e30f-405d-b5cf-be0d45cdee6e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:42.382517 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.382217    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f127fc10-e30f-405d-b5cf-be0d45cdee6e-kube-api-access-jvlj2" pod "f127fc10-e30f-405d-b5cf-be0d45cdee6e" (UID: "f127fc10-e30f-405d-b5cf-be0d45cdee6e"). InnerVolumeSpecName "kube-api-access-jvlj2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:42.445970 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.445935    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"
Oct 30 19:46:42.480433 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.480404    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-tmp-dir\") pod \"c3f73773-6c2f-47e4-a5a4-c57916f64ac9\" (UID: \"c3f73773-6c2f-47e4-a5a4-c57916f64ac9\") "
Oct 30 19:46:42.480576 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.480514    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-home\" (UniqueName: \"kubernetes.io/empty-dir/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-home\") pod \"c3f73773-6c2f-47e4-a5a4-c57916f64ac9\" (UID: \"c3f73773-6c2f-47e4-a5a4-c57916f64ac9\") "
Oct 30 19:46:42.480576 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.480566    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-tls-certs\" (UniqueName: \"kubernetes.io/secret/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-tls-certs\") pod \"c3f73773-6c2f-47e4-a5a4-c57916f64ac9\" (UID: \"c3f73773-6c2f-47e4-a5a4-c57916f64ac9\") "
Oct 30 19:46:42.480671 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.480635    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-dshm\") pod \"c3f73773-6c2f-47e4-a5a4-c57916f64ac9\" (UID: \"c3f73773-6c2f-47e4-a5a4-c57916f64ac9\") "
Oct 30 19:46:42.480740 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.480665    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-tmp-dir" pod "c3f73773-6c2f-47e4-a5a4-c57916f64ac9" (UID: "c3f73773-6c2f-47e4-a5a4-c57916f64ac9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:42.480740 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.480701    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-kube-api-access-qnjd6\" (UniqueName: \"kubernetes.io/projected/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-kube-api-access-qnjd6\") pod \"c3f73773-6c2f-47e4-a5a4-c57916f64ac9\" (UID: \"c3f73773-6c2f-47e4-a5a4-c57916f64ac9\") "
Oct 30 19:46:42.480907 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.480765    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-model-cache\") pod \"c3f73773-6c2f-47e4-a5a4-c57916f64ac9\" (UID: \"c3f73773-6c2f-47e4-a5a4-c57916f64ac9\") "
Oct 30 19:46:42.480907 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.480784    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-home" pod "c3f73773-6c2f-47e4-a5a4-c57916f64ac9" (UID: "c3f73773-6c2f-47e4-a5a4-c57916f64ac9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:42.481106 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.481082    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-model-cache" pod "c3f73773-6c2f-47e4-a5a4-c57916f64ac9" (UID: "c3f73773-6c2f-47e4-a5a4-c57916f64ac9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:42.481187 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.481162    5907 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f127fc10-e30f-405d-b5cf-be0d45cdee6e-tls-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:42.481187 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.481179    5907 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f127fc10-e30f-405d-b5cf-be0d45cdee6e-dshm\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:42.481285 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.481187    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvlj2\" (UniqueName: \"kubernetes.io/projected/f127fc10-e30f-405d-b5cf-be0d45cdee6e-kube-api-access-jvlj2\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:42.481285 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.481197    5907 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-tmp-dir\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:42.481285 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.481208    5907 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-home\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:42.483027 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.483003    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-dshm" pod "c3f73773-6c2f-47e4-a5a4-c57916f64ac9" (UID: "c3f73773-6c2f-47e4-a5a4-c57916f64ac9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:42.483112 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.483062    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-tls-certs" pod "c3f73773-6c2f-47e4-a5a4-c57916f64ac9" (UID: "c3f73773-6c2f-47e4-a5a4-c57916f64ac9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:46:42.483159 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.483147    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-kube-api-access-qnjd6" pod "c3f73773-6c2f-47e4-a5a4-c57916f64ac9" (UID: "c3f73773-6c2f-47e4-a5a4-c57916f64ac9"). InnerVolumeSpecName "kube-api-access-qnjd6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:42.582993 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.582961    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnjd6\" (UniqueName: \"kubernetes.io/projected/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-kube-api-access-qnjd6\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:42.582993 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.582989    5907 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-model-cache\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:42.582993 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.582998    5907 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-tls-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:42.582993 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.583006    5907 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3f73773-6c2f-47e4-a5a4-c57916f64ac9-dshm\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:42.890162 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.883237    5907 generic.go:356] "Generic (PLEG): container finished" podID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerID="02ae6121b780eea9dbffa8ba7a0ce00e619740331de6c77558feb4f596bf90f5" exitCode=0
Oct 30 19:46:42.890162 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.883538    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" event={"ID":"f127fc10-e30f-405d-b5cf-be0d45cdee6e","Type":"ContainerDied","Data":"02ae6121b780eea9dbffa8ba7a0ce00e619740331de6c77558feb4f596bf90f5"}
Oct 30 19:46:42.890162 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.883588    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8" event={"ID":"f127fc10-e30f-405d-b5cf-be0d45cdee6e","Type":"ContainerDied","Data":"a26d7e7fcfb838edb8f326693dfd0e73cddf5578933a37759603b195745a8e60"}
Oct 30 19:46:42.890162 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.883616    5907 scope.go:122] "RemoveContainer" containerID="02ae6121b780eea9dbffa8ba7a0ce00e619740331de6c77558feb4f596bf90f5"
Oct 30 19:46:42.890162 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.883874    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"
Oct 30 19:46:42.933180 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.933136    5907 generic.go:356] "Generic (PLEG): container finished" podID="c3f73773-6c2f-47e4-a5a4-c57916f64ac9" containerID="73be7a140f7c340a40c5680df706a9beff4c8be230f47f0ee20ffb1ec73d3071" exitCode=0
Oct 30 19:46:42.933333 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.933279    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6" event={"ID":"c3f73773-6c2f-47e4-a5a4-c57916f64ac9","Type":"ContainerDied","Data":"73be7a140f7c340a40c5680df706a9beff4c8be230f47f0ee20ffb1ec73d3071"}
Oct 30 19:46:42.933395 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.933320    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6" event={"ID":"c3f73773-6c2f-47e4-a5a4-c57916f64ac9","Type":"ContainerDied","Data":"04d95f1cf9c35a651b50e1052b3b5b43ee28c35868e65b5013ad53225dd94c92"}
Oct 30 19:46:42.933458 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.933416    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"
Oct 30 19:46:42.964753 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.964725    5907 scope.go:122] "RemoveContainer" containerID="02ae6121b780eea9dbffa8ba7a0ce00e619740331de6c77558feb4f596bf90f5"
Oct 30 19:46:42.966479 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:46:42.966155    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02ae6121b780eea9dbffa8ba7a0ce00e619740331de6c77558feb4f596bf90f5\": container with ID starting with 02ae6121b780eea9dbffa8ba7a0ce00e619740331de6c77558feb4f596bf90f5 not found: ID does not exist" containerID="02ae6121b780eea9dbffa8ba7a0ce00e619740331de6c77558feb4f596bf90f5"
Oct 30 19:46:42.966479 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.966196    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02ae6121b780eea9dbffa8ba7a0ce00e619740331de6c77558feb4f596bf90f5"} err="failed to get container status \"02ae6121b780eea9dbffa8ba7a0ce00e619740331de6c77558feb4f596bf90f5\": rpc error: code = NotFound desc = could not find container \"02ae6121b780eea9dbffa8ba7a0ce00e619740331de6c77558feb4f596bf90f5\": container with ID starting with 02ae6121b780eea9dbffa8ba7a0ce00e619740331de6c77558feb4f596bf90f5 not found: ID does not exist"
Oct 30 19:46:42.966479 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.966221    5907 scope.go:122] "RemoveContainer" containerID="73be7a140f7c340a40c5680df706a9beff4c8be230f47f0ee20ffb1ec73d3071"
Oct 30 19:46:42.995619 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.995587    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"]
Oct 30 19:46:42.998632 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.998602    5907 scope.go:122] "RemoveContainer" containerID="73be7a140f7c340a40c5680df706a9beff4c8be230f47f0ee20ffb1ec73d3071"
Oct 30 19:46:42.999038 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:46:42.999016    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73be7a140f7c340a40c5680df706a9beff4c8be230f47f0ee20ffb1ec73d3071\": container with ID starting with 73be7a140f7c340a40c5680df706a9beff4c8be230f47f0ee20ffb1ec73d3071 not found: ID does not exist" containerID="73be7a140f7c340a40c5680df706a9beff4c8be230f47f0ee20ffb1ec73d3071"
Oct 30 19:46:42.999127 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:42.999046    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73be7a140f7c340a40c5680df706a9beff4c8be230f47f0ee20ffb1ec73d3071"} err="failed to get container status \"73be7a140f7c340a40c5680df706a9beff4c8be230f47f0ee20ffb1ec73d3071\": rpc error: code = NotFound desc = could not find container \"73be7a140f7c340a40c5680df706a9beff4c8be230f47f0ee20ffb1ec73d3071\": container with ID starting with 73be7a140f7c340a40c5680df706a9beff4c8be230f47f0ee20ffb1ec73d3071 not found: ID does not exist"
Oct 30 19:46:43.006385 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:43.006354    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-4b4c34/premium-w0-4b4c34-kserve-5d84599686-zgzf8"]
Oct 30 19:46:43.036952 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:43.036921    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"]
Oct 30 19:46:43.043340 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:43.043303    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-67953efd/e2e-lifecycle-model-67953efd-kserve-84c6499b69-5vgn6"]
Oct 30 19:46:44.288427 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:44.288392    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c3f73773-6c2f-47e4-a5a4-c57916f64ac9" path="/var/lib/kubelet/pods/c3f73773-6c2f-47e4-a5a4-c57916f64ac9/volumes"
Oct 30 19:46:44.288919 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:44.288904    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" path="/var/lib/kubelet/pods/f127fc10-e30f-405d-b5cf-be0d45cdee6e/volumes"
Oct 30 19:46:47.174189 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.174138    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-9de441-ffb9d7698-7bgvb" podUID="40e15386-d751-4b28-8124-a042609e0fd1" containerName="payload-processing" containerID="cri-o://b6631875f83ead306878173b8e367d433836e80d0694971ee92eb1b8d7ca42e1" gracePeriod=25
Oct 30 19:46:47.207682 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.207644    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9de441-67cfb6c4c5-vgswm" podUID="a033862c-f1bf-4331-b9dc-ebfa073c5caf" containerName="payload-pre-processing" containerID="cri-o://069ab9d4fac1c81d11a29be7aa25506649cf9a23452815ad58f37dd3869ae8b1" gracePeriod=25
Oct 30 19:46:47.214599 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.214566    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-967ecdfb-8qbwl"]
Oct 30 19:46:47.215250 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.215235    5907 cpu_manager.go:414] "containerMap: removing container" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main"
Oct 30 19:46:47.215317 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.215252    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main"
Oct 30 19:46:47.215317 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.215298    5907 cpu_manager.go:414] "containerMap: removing container" podUID="c3f73773-6c2f-47e4-a5a4-c57916f64ac9" containerName="main"
Oct 30 19:46:47.215317 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.215305    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c3f73773-6c2f-47e4-a5a4-c57916f64ac9" containerName="main"
Oct 30 19:46:47.215417 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.215320    5907 cpu_manager.go:414] "containerMap: removing container" podUID="0ddee3f4-7acc-44ed-affc-d0096c44b37a" containerName="revoke-keys"
Oct 30 19:46:47.215417 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.215326    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0ddee3f4-7acc-44ed-affc-d0096c44b37a" containerName="revoke-keys"
Oct 30 19:46:47.215417 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.215414    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="c3f73773-6c2f-47e4-a5a4-c57916f64ac9" containerName="main"
Oct 30 19:46:47.215554 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.215423    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="0ddee3f4-7acc-44ed-affc-d0096c44b37a" containerName="revoke-keys"
Oct 30 19:46:47.215554 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.215436    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="f127fc10-e30f-405d-b5cf-be0d45cdee6e" containerName="main"
Oct 30 19:46:47.215554 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.215446    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="b5678ec4-1e40-4b66-a9d2-7e6bf08db0bf" containerName="istio-proxy"
Oct 30 19:46:47.219532 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.219515    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-967ecdfb-8qbwl"
Oct 30 19:46:47.251596 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.251572    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:46:47.257687 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.257663    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:46:47.276675 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.276637    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9eb15025-db65-4deb-8a88-d6302ab53996-maas-api-xks-ca\") pod \"maas-api-revoke-sub-967ecdfb-8qbwl\" (UID: \"9eb15025-db65-4deb-8a88-d6302ab53996\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-967ecdfb-8qbwl"
Oct 30 19:46:47.276843 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.276773    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9eb15025-db65-4deb-8a88-d6302ab53996-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-967ecdfb-8qbwl\" (UID: \"9eb15025-db65-4deb-8a88-d6302ab53996\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-967ecdfb-8qbwl"
Oct 30 19:46:47.276931 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.276840    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4b2n\" (UniqueName: \"kubernetes.io/projected/9eb15025-db65-4deb-8a88-d6302ab53996-kube-api-access-z4b2n\") pod \"maas-api-revoke-sub-967ecdfb-8qbwl\" (UID: \"9eb15025-db65-4deb-8a88-d6302ab53996\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-967ecdfb-8qbwl"
Oct 30 19:46:47.309616 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.309570    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-967ecdfb-8qbwl"]
Oct 30 19:46:47.339700 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.339668    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b8eda0f9-59xt4"]
Oct 30 19:46:47.344544 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.344514    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8eda0f9-59xt4"
Oct 30 19:46:47.379692 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.379644    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9eb15025-db65-4deb-8a88-d6302ab53996-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-967ecdfb-8qbwl\" (UID: \"9eb15025-db65-4deb-8a88-d6302ab53996\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-967ecdfb-8qbwl"
Oct 30 19:46:47.379895 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.379763    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4b2n\" (UniqueName: \"kubernetes.io/projected/9eb15025-db65-4deb-8a88-d6302ab53996-kube-api-access-z4b2n\") pod \"maas-api-revoke-sub-967ecdfb-8qbwl\" (UID: \"9eb15025-db65-4deb-8a88-d6302ab53996\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-967ecdfb-8qbwl"
Oct 30 19:46:47.379895 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.379821    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7r525\" (UniqueName: \"kubernetes.io/projected/e3eabdcb-14c1-446c-88d9-261df5e254cb-kube-api-access-7r525\") pod \"maas-api-revoke-sub-b8eda0f9-59xt4\" (UID: \"e3eabdcb-14c1-446c-88d9-261df5e254cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8eda0f9-59xt4"
Oct 30 19:46:47.380028 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.379902    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e3eabdcb-14c1-446c-88d9-261df5e254cb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b8eda0f9-59xt4\" (UID: \"e3eabdcb-14c1-446c-88d9-261df5e254cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8eda0f9-59xt4"
Oct 30 19:46:47.380355 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.380118    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e3eabdcb-14c1-446c-88d9-261df5e254cb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b8eda0f9-59xt4\" (UID: \"e3eabdcb-14c1-446c-88d9-261df5e254cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8eda0f9-59xt4"
Oct 30 19:46:47.380355 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.380204    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9eb15025-db65-4deb-8a88-d6302ab53996-maas-api-xks-ca\") pod \"maas-api-revoke-sub-967ecdfb-8qbwl\" (UID: \"9eb15025-db65-4deb-8a88-d6302ab53996\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-967ecdfb-8qbwl"
Oct 30 19:46:47.381245 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.381146    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9eb15025-db65-4deb-8a88-d6302ab53996-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-967ecdfb-8qbwl\" (UID: \"9eb15025-db65-4deb-8a88-d6302ab53996\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-967ecdfb-8qbwl"
Oct 30 19:46:47.383630 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.383604    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9eb15025-db65-4deb-8a88-d6302ab53996-maas-api-xks-ca\") pod \"maas-api-revoke-sub-967ecdfb-8qbwl\" (UID: \"9eb15025-db65-4deb-8a88-d6302ab53996\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-967ecdfb-8qbwl"
Oct 30 19:46:47.399786 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.399743    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b8eda0f9-59xt4"]
Oct 30 19:46:47.413770 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.413734    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4b2n\" (UniqueName: \"kubernetes.io/projected/9eb15025-db65-4deb-8a88-d6302ab53996-kube-api-access-z4b2n\") pod \"maas-api-revoke-sub-967ecdfb-8qbwl\" (UID: \"9eb15025-db65-4deb-8a88-d6302ab53996\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-967ecdfb-8qbwl"
Oct 30 19:46:47.483844 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.481225    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7r525\" (UniqueName: \"kubernetes.io/projected/e3eabdcb-14c1-446c-88d9-261df5e254cb-kube-api-access-7r525\") pod \"maas-api-revoke-sub-b8eda0f9-59xt4\" (UID: \"e3eabdcb-14c1-446c-88d9-261df5e254cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8eda0f9-59xt4"
Oct 30 19:46:47.483844 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.481284    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e3eabdcb-14c1-446c-88d9-261df5e254cb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b8eda0f9-59xt4\" (UID: \"e3eabdcb-14c1-446c-88d9-261df5e254cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8eda0f9-59xt4"
Oct 30 19:46:47.483844 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.481330    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e3eabdcb-14c1-446c-88d9-261df5e254cb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b8eda0f9-59xt4\" (UID: \"e3eabdcb-14c1-446c-88d9-261df5e254cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8eda0f9-59xt4"
Oct 30 19:46:47.486164 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.484572    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e3eabdcb-14c1-446c-88d9-261df5e254cb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b8eda0f9-59xt4\" (UID: \"e3eabdcb-14c1-446c-88d9-261df5e254cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8eda0f9-59xt4"
Oct 30 19:46:47.488734 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.488701    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e3eabdcb-14c1-446c-88d9-261df5e254cb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b8eda0f9-59xt4\" (UID: \"e3eabdcb-14c1-446c-88d9-261df5e254cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8eda0f9-59xt4"
Oct 30 19:46:47.517107 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:46:47.517037    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:46:47.526364 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.526334    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7r525\" (UniqueName: \"kubernetes.io/projected/e3eabdcb-14c1-446c-88d9-261df5e254cb-kube-api-access-7r525\") pod \"maas-api-revoke-sub-b8eda0f9-59xt4\" (UID: \"e3eabdcb-14c1-446c-88d9-261df5e254cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8eda0f9-59xt4"
Oct 30 19:46:47.536305 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.536280    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-9de441-ffb9d7698-7bgvb"
Oct 30 19:46:47.538708 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.538684    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9de441-67cfb6c4c5-vgswm"
Oct 30 19:46:47.582673 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.582633    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a033862c-f1bf-4331-b9dc-ebfa073c5caf-kube-api-access-29s4n\" (UniqueName: \"kubernetes.io/projected/a033862c-f1bf-4331-b9dc-ebfa073c5caf-kube-api-access-29s4n\") pod \"a033862c-f1bf-4331-b9dc-ebfa073c5caf\" (UID: \"a033862c-f1bf-4331-b9dc-ebfa073c5caf\") "
Oct 30 19:46:47.582935 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.582757    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/40e15386-d751-4b28-8124-a042609e0fd1-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40e15386-d751-4b28-8124-a042609e0fd1-plugins-config-volume\") pod \"40e15386-d751-4b28-8124-a042609e0fd1\" (UID: \"40e15386-d751-4b28-8124-a042609e0fd1\") "
Oct 30 19:46:47.582935 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.582822    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/40e15386-d751-4b28-8124-a042609e0fd1-kube-api-access-drg6m\" (UniqueName: \"kubernetes.io/projected/40e15386-d751-4b28-8124-a042609e0fd1-kube-api-access-drg6m\") pod \"40e15386-d751-4b28-8124-a042609e0fd1\" (UID: \"40e15386-d751-4b28-8124-a042609e0fd1\") "
Oct 30 19:46:47.582935 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.582909    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a033862c-f1bf-4331-b9dc-ebfa073c5caf-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a033862c-f1bf-4331-b9dc-ebfa073c5caf-plugins-config-volume\") pod \"a033862c-f1bf-4331-b9dc-ebfa073c5caf\" (UID: \"a033862c-f1bf-4331-b9dc-ebfa073c5caf\") "
Oct 30 19:46:47.583241 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.583214    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40e15386-d751-4b28-8124-a042609e0fd1-plugins-config-volume" pod "40e15386-d751-4b28-8124-a042609e0fd1" (UID: "40e15386-d751-4b28-8124-a042609e0fd1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:47.583480 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.583456    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a033862c-f1bf-4331-b9dc-ebfa073c5caf-plugins-config-volume" pod "a033862c-f1bf-4331-b9dc-ebfa073c5caf" (UID: "a033862c-f1bf-4331-b9dc-ebfa073c5caf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:47.583540 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.583515    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40e15386-d751-4b28-8124-a042609e0fd1-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:47.585683 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.585660    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40e15386-d751-4b28-8124-a042609e0fd1-kube-api-access-drg6m" pod "40e15386-d751-4b28-8124-a042609e0fd1" (UID: "40e15386-d751-4b28-8124-a042609e0fd1"). InnerVolumeSpecName "kube-api-access-drg6m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:47.586555 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.586467    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a033862c-f1bf-4331-b9dc-ebfa073c5caf-kube-api-access-29s4n" pod "a033862c-f1bf-4331-b9dc-ebfa073c5caf" (UID: "a033862c-f1bf-4331-b9dc-ebfa073c5caf"). InnerVolumeSpecName "kube-api-access-29s4n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:47.650016 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.649984    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-967ecdfb-8qbwl"
Oct 30 19:46:47.664374 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.664349    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8eda0f9-59xt4"
Oct 30 19:46:47.685485 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.685452    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drg6m\" (UniqueName: \"kubernetes.io/projected/40e15386-d751-4b28-8124-a042609e0fd1-kube-api-access-drg6m\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:47.685485 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.685489    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a033862c-f1bf-4331-b9dc-ebfa073c5caf-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:47.685711 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.685505    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29s4n\" (UniqueName: \"kubernetes.io/projected/a033862c-f1bf-4331-b9dc-ebfa073c5caf-kube-api-access-29s4n\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:47.848895 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:47.848829    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-fe4ec3-7944b894b6-6qx48"]
Oct 30 19:46:48.003639 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:48.003603    5907 generic.go:356] "Generic (PLEG): container finished" podID="a033862c-f1bf-4331-b9dc-ebfa073c5caf" containerID="069ab9d4fac1c81d11a29be7aa25506649cf9a23452815ad58f37dd3869ae8b1" exitCode=0
Oct 30 19:46:48.003996 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:48.003754    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9de441-67cfb6c4c5-vgswm"
Oct 30 19:46:48.004147 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:48.003780    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9de441-67cfb6c4c5-vgswm" event={"ID":"a033862c-f1bf-4331-b9dc-ebfa073c5caf","Type":"ContainerDied","Data":"069ab9d4fac1c81d11a29be7aa25506649cf9a23452815ad58f37dd3869ae8b1"}
Oct 30 19:46:48.004147 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:48.004100    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9de441-67cfb6c4c5-vgswm" event={"ID":"a033862c-f1bf-4331-b9dc-ebfa073c5caf","Type":"ContainerDied","Data":"b30898ed4b3451b14d87237697f3fc0a851730b114e08d4e627063ba8688ed96"}
Oct 30 19:46:48.004147 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:48.004146    5907 scope.go:122] "RemoveContainer" containerID="069ab9d4fac1c81d11a29be7aa25506649cf9a23452815ad58f37dd3869ae8b1"
Oct 30 19:46:48.007439 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:48.007404    5907 generic.go:356] "Generic (PLEG): container finished" podID="40e15386-d751-4b28-8124-a042609e0fd1" containerID="b6631875f83ead306878173b8e367d433836e80d0694971ee92eb1b8d7ca42e1" exitCode=0
Oct 30 19:46:48.007634 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:48.007465    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-9de441-ffb9d7698-7bgvb" event={"ID":"40e15386-d751-4b28-8124-a042609e0fd1","Type":"ContainerDied","Data":"b6631875f83ead306878173b8e367d433836e80d0694971ee92eb1b8d7ca42e1"}
Oct 30 19:46:48.007805 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:48.007777    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-9de441-ffb9d7698-7bgvb" event={"ID":"40e15386-d751-4b28-8124-a042609e0fd1","Type":"ContainerDied","Data":"f254b0e27b5e0cb9a61314a0d5d7b75c7c2ebfb7e737694fa687980b8909fdec"}
Oct 30 19:46:48.008014 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:48.007555    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-9de441-ffb9d7698-7bgvb"
Oct 30 19:46:48.019734 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:48.019707    5907 scope.go:122] "RemoveContainer" containerID="069ab9d4fac1c81d11a29be7aa25506649cf9a23452815ad58f37dd3869ae8b1"
Oct 30 19:46:48.020379 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:46:48.020342    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"069ab9d4fac1c81d11a29be7aa25506649cf9a23452815ad58f37dd3869ae8b1\": container with ID starting with 069ab9d4fac1c81d11a29be7aa25506649cf9a23452815ad58f37dd3869ae8b1 not found: ID does not exist" containerID="069ab9d4fac1c81d11a29be7aa25506649cf9a23452815ad58f37dd3869ae8b1"
Oct 30 19:46:48.020450 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:48.020383    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"069ab9d4fac1c81d11a29be7aa25506649cf9a23452815ad58f37dd3869ae8b1"} err="failed to get container status \"069ab9d4fac1c81d11a29be7aa25506649cf9a23452815ad58f37dd3869ae8b1\": rpc error: code = NotFound desc = could not find container \"069ab9d4fac1c81d11a29be7aa25506649cf9a23452815ad58f37dd3869ae8b1\": container with ID starting with 069ab9d4fac1c81d11a29be7aa25506649cf9a23452815ad58f37dd3869ae8b1 not found: ID does not exist"
Oct 30 19:46:48.020450 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:48.020409    5907 scope.go:122] "RemoveContainer" containerID="b6631875f83ead306878173b8e367d433836e80d0694971ee92eb1b8d7ca42e1"
Oct 30 19:46:48.031290 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:48.031257    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-967ecdfb-8qbwl"]
Oct 30 19:46:48.035117 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:48.035087    5907 scope.go:122] "RemoveContainer" containerID="b6631875f83ead306878173b8e367d433836e80d0694971ee92eb1b8d7ca42e1"
Oct 30 19:46:48.035445 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:46:48.035426    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6631875f83ead306878173b8e367d433836e80d0694971ee92eb1b8d7ca42e1\": container with ID starting with b6631875f83ead306878173b8e367d433836e80d0694971ee92eb1b8d7ca42e1 not found: ID does not exist" containerID="b6631875f83ead306878173b8e367d433836e80d0694971ee92eb1b8d7ca42e1"
Oct 30 19:46:48.035504 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:48.035455    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6631875f83ead306878173b8e367d433836e80d0694971ee92eb1b8d7ca42e1"} err="failed to get container status \"b6631875f83ead306878173b8e367d433836e80d0694971ee92eb1b8d7ca42e1\": rpc error: code = NotFound desc = could not find container \"b6631875f83ead306878173b8e367d433836e80d0694971ee92eb1b8d7ca42e1\": container with ID starting with b6631875f83ead306878173b8e367d433836e80d0694971ee92eb1b8d7ca42e1 not found: ID does not exist"
Oct 30 19:46:48.037115 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:46:48.037090    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eb15025_db65_4deb_8a88_d6302ab53996.slice/crio-cce3f54e704724678f0b2506e56588a63bf169dd5fa7ab88e3c6d4bf91e6e34b WatchSource:0}: Error finding container cce3f54e704724678f0b2506e56588a63bf169dd5fa7ab88e3c6d4bf91e6e34b: Status 404 returned error can't find the container with id cce3f54e704724678f0b2506e56588a63bf169dd5fa7ab88e3c6d4bf91e6e34b
Oct 30 19:46:48.071877 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:48.071826    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9de441-ffb9d7698-7bgvb"]
Oct 30 19:46:48.095024 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:48.094981    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b8eda0f9-59xt4"]
Oct 30 19:46:48.102939 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:46:48.102891    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3eabdcb_14c1_446c_88d9_261df5e254cb.slice/crio-399b81bdb3cdaed462a3aae7d6e35ef76801f878854ea4000acb338e14c23282 WatchSource:0}: Error finding container 399b81bdb3cdaed462a3aae7d6e35ef76801f878854ea4000acb338e14c23282: Status 404 returned error can't find the container with id 399b81bdb3cdaed462a3aae7d6e35ef76801f878854ea4000acb338e14c23282
Oct 30 19:46:48.105082 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:48.105027    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9de441-ffb9d7698-7bgvb"]
Oct 30 19:46:48.160442 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:48.160366    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9de441-67cfb6c4c5-vgswm"]
Oct 30 19:46:48.165777 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:48.165681    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9de441-67cfb6c4c5-vgswm"]
Oct 30 19:46:48.290969 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:48.290848    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="40e15386-d751-4b28-8124-a042609e0fd1" path="/var/lib/kubelet/pods/40e15386-d751-4b28-8124-a042609e0fd1/volumes"
Oct 30 19:46:48.291423 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:48.291406    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a033862c-f1bf-4331-b9dc-ebfa073c5caf" path="/var/lib/kubelet/pods/a033862c-f1bf-4331-b9dc-ebfa073c5caf/volumes"
Oct 30 19:46:49.027790 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:49.027751    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-967ecdfb-8qbwl" event={"ID":"9eb15025-db65-4deb-8a88-d6302ab53996","Type":"ContainerStarted","Data":"8886ea31c61d5c833b77552dd9a7e8acf584ceced29aa73d466f42a52fb392ea"}
Oct 30 19:46:49.028084 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:49.027818    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-967ecdfb-8qbwl" event={"ID":"9eb15025-db65-4deb-8a88-d6302ab53996","Type":"ContainerStarted","Data":"cce3f54e704724678f0b2506e56588a63bf169dd5fa7ab88e3c6d4bf91e6e34b"}
Oct 30 19:46:49.030782 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:49.030426    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8eda0f9-59xt4" event={"ID":"e3eabdcb-14c1-446c-88d9-261df5e254cb","Type":"ContainerStarted","Data":"b25d3d304895da9e40e246b343fb784589274c89a57815380cb65691cdbaf0d8"}
Oct 30 19:46:49.031066 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:49.031030    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8eda0f9-59xt4" event={"ID":"e3eabdcb-14c1-446c-88d9-261df5e254cb","Type":"ContainerStarted","Data":"399b81bdb3cdaed462a3aae7d6e35ef76801f878854ea4000acb338e14c23282"}
Oct 30 19:46:49.074586 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:49.074527    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-967ecdfb-8qbwl" podStartSLOduration=2.074512623 podStartE2EDuration="2.074512623s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:46:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:46:49.07117495 +0000 UTC m=+1520.914106890" watchObservedRunningTime="2026-10-30 19:46:49.074512623 +0000 UTC m=+1520.917444562"
Oct 30 19:46:49.118407 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:49.118343    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8eda0f9-59xt4" podStartSLOduration=2.118326354 podStartE2EDuration="2.118326354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:46:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:46:49.114122707 +0000 UTC m=+1520.957054648" watchObservedRunningTime="2026-10-30 19:46:49.118326354 +0000 UTC m=+1520.961258293"
Oct 30 19:46:50.042945 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:50.042897    5907 generic.go:356] "Generic (PLEG): container finished" podID="9eb15025-db65-4deb-8a88-d6302ab53996" containerID="8886ea31c61d5c833b77552dd9a7e8acf584ceced29aa73d466f42a52fb392ea" exitCode=0
Oct 30 19:46:50.043464 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:50.043023    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-967ecdfb-8qbwl" event={"ID":"9eb15025-db65-4deb-8a88-d6302ab53996","Type":"ContainerDied","Data":"8886ea31c61d5c833b77552dd9a7e8acf584ceced29aa73d466f42a52fb392ea"}
Oct 30 19:46:50.045094 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:50.045063    5907 generic.go:356] "Generic (PLEG): container finished" podID="e3eabdcb-14c1-446c-88d9-261df5e254cb" containerID="b25d3d304895da9e40e246b343fb784589274c89a57815380cb65691cdbaf0d8" exitCode=0
Oct 30 19:46:50.045245 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:50.045099    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8eda0f9-59xt4" event={"ID":"e3eabdcb-14c1-446c-88d9-261df5e254cb","Type":"ContainerDied","Data":"b25d3d304895da9e40e246b343fb784589274c89a57815380cb65691cdbaf0d8"}
Oct 30 19:46:50.052308 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:50.052281    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-4b4c34-5bb88fdc94-s44w4" podUID="2dee0a84-130e-413d-8294-a86bf9c82c92" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:46:50.052595 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:50.052569    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b4c34-556f66b87f-d864z" podUID="b9fbde13-05e2-47a9-93bc-65cbd47efa6a" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:46:51.259829 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:51.259795    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-967ecdfb-8qbwl"
Oct 30 19:46:51.301961 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:51.301936    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8eda0f9-59xt4"
Oct 30 19:46:51.388476 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:51.388442    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e3eabdcb-14c1-446c-88d9-261df5e254cb-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e3eabdcb-14c1-446c-88d9-261df5e254cb-maas-api-xks-ca\") pod \"e3eabdcb-14c1-446c-88d9-261df5e254cb\" (UID: \"e3eabdcb-14c1-446c-88d9-261df5e254cb\") "
Oct 30 19:46:51.388664 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:51.388596    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e3eabdcb-14c1-446c-88d9-261df5e254cb-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e3eabdcb-14c1-446c-88d9-261df5e254cb-maas-api-openshift-ca\") pod \"e3eabdcb-14c1-446c-88d9-261df5e254cb\" (UID: \"e3eabdcb-14c1-446c-88d9-261df5e254cb\") "
Oct 30 19:46:51.388664 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:51.388648    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e3eabdcb-14c1-446c-88d9-261df5e254cb-kube-api-access-7r525\" (UniqueName: \"kubernetes.io/projected/e3eabdcb-14c1-446c-88d9-261df5e254cb-kube-api-access-7r525\") pod \"e3eabdcb-14c1-446c-88d9-261df5e254cb\" (UID: \"e3eabdcb-14c1-446c-88d9-261df5e254cb\") "
Oct 30 19:46:51.388785 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:51.388689    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9eb15025-db65-4deb-8a88-d6302ab53996-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9eb15025-db65-4deb-8a88-d6302ab53996-maas-api-openshift-ca\") pod \"9eb15025-db65-4deb-8a88-d6302ab53996\" (UID: \"9eb15025-db65-4deb-8a88-d6302ab53996\") "
Oct 30 19:46:51.388785 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:51.388731    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9eb15025-db65-4deb-8a88-d6302ab53996-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9eb15025-db65-4deb-8a88-d6302ab53996-maas-api-xks-ca\") pod \"9eb15025-db65-4deb-8a88-d6302ab53996\" (UID: \"9eb15025-db65-4deb-8a88-d6302ab53996\") "
Oct 30 19:46:51.389289 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:51.388794    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9eb15025-db65-4deb-8a88-d6302ab53996-kube-api-access-z4b2n\" (UniqueName: \"kubernetes.io/projected/9eb15025-db65-4deb-8a88-d6302ab53996-kube-api-access-z4b2n\") pod \"9eb15025-db65-4deb-8a88-d6302ab53996\" (UID: \"9eb15025-db65-4deb-8a88-d6302ab53996\") "
Oct 30 19:46:51.389289 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:51.389061    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3eabdcb-14c1-446c-88d9-261df5e254cb-maas-api-openshift-ca" pod "e3eabdcb-14c1-446c-88d9-261df5e254cb" (UID: "e3eabdcb-14c1-446c-88d9-261df5e254cb"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:51.389289 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:51.389109    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9eb15025-db65-4deb-8a88-d6302ab53996-maas-api-openshift-ca" pod "9eb15025-db65-4deb-8a88-d6302ab53996" (UID: "9eb15025-db65-4deb-8a88-d6302ab53996"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:51.389680 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:51.389641    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e3eabdcb-14c1-446c-88d9-261df5e254cb-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:51.389784 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:51.389681    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9eb15025-db65-4deb-8a88-d6302ab53996-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:51.391200 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:51.391166    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3eabdcb-14c1-446c-88d9-261df5e254cb-kube-api-access-7r525" pod "e3eabdcb-14c1-446c-88d9-261df5e254cb" (UID: "e3eabdcb-14c1-446c-88d9-261df5e254cb"). InnerVolumeSpecName "kube-api-access-7r525". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:51.391585 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:51.391562    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9eb15025-db65-4deb-8a88-d6302ab53996-maas-api-xks-ca" pod "9eb15025-db65-4deb-8a88-d6302ab53996" (UID: "9eb15025-db65-4deb-8a88-d6302ab53996"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:46:51.391666 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:51.391574    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3eabdcb-14c1-446c-88d9-261df5e254cb-maas-api-xks-ca" pod "e3eabdcb-14c1-446c-88d9-261df5e254cb" (UID: "e3eabdcb-14c1-446c-88d9-261df5e254cb"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:46:51.391731 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:51.391688    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9eb15025-db65-4deb-8a88-d6302ab53996-kube-api-access-z4b2n" pod "9eb15025-db65-4deb-8a88-d6302ab53996" (UID: "9eb15025-db65-4deb-8a88-d6302ab53996"). InnerVolumeSpecName "kube-api-access-z4b2n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:51.490744 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:51.490713    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e3eabdcb-14c1-446c-88d9-261df5e254cb-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:51.490744 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:51.490743    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7r525\" (UniqueName: \"kubernetes.io/projected/e3eabdcb-14c1-446c-88d9-261df5e254cb-kube-api-access-7r525\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:51.491007 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:51.490753    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9eb15025-db65-4deb-8a88-d6302ab53996-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:51.491007 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:51.490762    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4b2n\" (UniqueName: \"kubernetes.io/projected/9eb15025-db65-4deb-8a88-d6302ab53996-kube-api-access-z4b2n\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:52.063041 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:52.063012    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-967ecdfb-8qbwl"
Oct 30 19:46:52.063266 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:52.063042    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-967ecdfb-8qbwl" event={"ID":"9eb15025-db65-4deb-8a88-d6302ab53996","Type":"ContainerDied","Data":"cce3f54e704724678f0b2506e56588a63bf169dd5fa7ab88e3c6d4bf91e6e34b"}
Oct 30 19:46:52.063266 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:52.063097    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cce3f54e704724678f0b2506e56588a63bf169dd5fa7ab88e3c6d4bf91e6e34b"
Oct 30 19:46:52.064615 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:52.064591    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8eda0f9-59xt4"
Oct 30 19:46:52.064749 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:52.064623    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8eda0f9-59xt4" event={"ID":"e3eabdcb-14c1-446c-88d9-261df5e254cb","Type":"ContainerDied","Data":"399b81bdb3cdaed462a3aae7d6e35ef76801f878854ea4000acb338e14c23282"}
Oct 30 19:46:52.064749 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:52.064663    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="399b81bdb3cdaed462a3aae7d6e35ef76801f878854ea4000acb338e14c23282"
Oct 30 19:46:52.858886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:52.858160    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe4ec3-7944b894b6-6qx48" podUID="e876e507-2b04-43fd-b788-b10d7d15748a" containerName="payload-processing" containerID="cri-o://97e542a8cc024aa83523d5e6c8634887ffa5e67220bbd716fd4c10d77fd76ff0" gracePeriod=25
Oct 30 19:46:53.075978 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:53.075930    5907 generic.go:356] "Generic (PLEG): container finished" podID="e876e507-2b04-43fd-b788-b10d7d15748a" containerID="97e542a8cc024aa83523d5e6c8634887ffa5e67220bbd716fd4c10d77fd76ff0" exitCode=0
Oct 30 19:46:53.076173 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:53.076141    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe4ec3-7944b894b6-6qx48" event={"ID":"e876e507-2b04-43fd-b788-b10d7d15748a","Type":"ContainerDied","Data":"97e542a8cc024aa83523d5e6c8634887ffa5e67220bbd716fd4c10d77fd76ff0"}
Oct 30 19:46:53.128397 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:53.128367    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-fe4ec3-jrzc5"]
Oct 30 19:46:53.140349 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:53.140321    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-fe4ec3-jrzc5"]
Oct 30 19:46:53.155587 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:53.155562    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe4ec3-7944b894b6-6qx48"
Oct 30 19:46:53.244981 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:53.227164    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e876e507-2b04-43fd-b788-b10d7d15748a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e876e507-2b04-43fd-b788-b10d7d15748a-plugins-config-volume\") pod \"e876e507-2b04-43fd-b788-b10d7d15748a\" (UID: \"e876e507-2b04-43fd-b788-b10d7d15748a\") "
Oct 30 19:46:53.244981 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:53.227355    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e876e507-2b04-43fd-b788-b10d7d15748a-kube-api-access-tnlgn\" (UniqueName: \"kubernetes.io/projected/e876e507-2b04-43fd-b788-b10d7d15748a-kube-api-access-tnlgn\") pod \"e876e507-2b04-43fd-b788-b10d7d15748a\" (UID: \"e876e507-2b04-43fd-b788-b10d7d15748a\") "
Oct 30 19:46:53.244981 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:53.231468    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e876e507-2b04-43fd-b788-b10d7d15748a-plugins-config-volume" pod "e876e507-2b04-43fd-b788-b10d7d15748a" (UID: "e876e507-2b04-43fd-b788-b10d7d15748a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:53.244981 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:53.242712    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e876e507-2b04-43fd-b788-b10d7d15748a-kube-api-access-tnlgn" pod "e876e507-2b04-43fd-b788-b10d7d15748a" (UID: "e876e507-2b04-43fd-b788-b10d7d15748a"). InnerVolumeSpecName "kube-api-access-tnlgn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:53.329280 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:53.329248    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tnlgn\" (UniqueName: \"kubernetes.io/projected/e876e507-2b04-43fd-b788-b10d7d15748a-kube-api-access-tnlgn\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:53.329280 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:53.329279    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e876e507-2b04-43fd-b788-b10d7d15748a-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:54.085204 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:54.085170    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe4ec3-7944b894b6-6qx48"
Oct 30 19:46:54.085658 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:54.085210    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe4ec3-7944b894b6-6qx48" event={"ID":"e876e507-2b04-43fd-b788-b10d7d15748a","Type":"ContainerDied","Data":"a4fa0df8c8fd0059ae3d0d231e0e0234919a473b07b60609b72cc288fd711b82"}
Oct 30 19:46:54.085658 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:54.085277    5907 scope.go:122] "RemoveContainer" containerID="97e542a8cc024aa83523d5e6c8634887ffa5e67220bbd716fd4c10d77fd76ff0"
Oct 30 19:46:54.141619 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:54.141588    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-fe4ec3-7944b894b6-6qx48"]
Oct 30 19:46:54.154892 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:54.154848    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-fe4ec3-7944b894b6-6qx48"]
Oct 30 19:46:54.289310 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:54.289279    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0ddee3f4-7acc-44ed-affc-d0096c44b37a" path="/var/lib/kubelet/pods/0ddee3f4-7acc-44ed-affc-d0096c44b37a/volumes"
Oct 30 19:46:54.289722 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:54.289705    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e876e507-2b04-43fd-b788-b10d7d15748a" path="/var/lib/kubelet/pods/e876e507-2b04-43fd-b788-b10d7d15748a/volumes"
Oct 30 19:46:58.128903 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:58.126445    5907 generic.go:356] "Generic (PLEG): container finished" podID="2dee0a84-130e-413d-8294-a86bf9c82c92" containerID="a312f454db6b8f05a75ca06ed4c412547765da1bc3784654cb3ecf916aada2b5" exitCode=0
Oct 30 19:46:58.128903 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:58.126694    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-4b4c34-5bb88fdc94-s44w4" event={"ID":"2dee0a84-130e-413d-8294-a86bf9c82c92","Type":"ContainerDied","Data":"a312f454db6b8f05a75ca06ed4c412547765da1bc3784654cb3ecf916aada2b5"}
Oct 30 19:46:58.314285 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:58.314255    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-4b4c34-5bb88fdc94-s44w4"
Oct 30 19:46:58.351829 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:58.351803    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b4c34-556f66b87f-d864z"
Oct 30 19:46:58.400096 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:58.400058    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2dee0a84-130e-413d-8294-a86bf9c82c92-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2dee0a84-130e-413d-8294-a86bf9c82c92-plugins-config-volume\") pod \"2dee0a84-130e-413d-8294-a86bf9c82c92\" (UID: \"2dee0a84-130e-413d-8294-a86bf9c82c92\") "
Oct 30 19:46:58.400271 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:58.400143    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b9fbde13-05e2-47a9-93bc-65cbd47efa6a-kube-api-access-znxqt\" (UniqueName: \"kubernetes.io/projected/b9fbde13-05e2-47a9-93bc-65cbd47efa6a-kube-api-access-znxqt\") pod \"b9fbde13-05e2-47a9-93bc-65cbd47efa6a\" (UID: \"b9fbde13-05e2-47a9-93bc-65cbd47efa6a\") "
Oct 30 19:46:58.400271 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:58.400213    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2dee0a84-130e-413d-8294-a86bf9c82c92-kube-api-access-bbj8w\" (UniqueName: \"kubernetes.io/projected/2dee0a84-130e-413d-8294-a86bf9c82c92-kube-api-access-bbj8w\") pod \"2dee0a84-130e-413d-8294-a86bf9c82c92\" (UID: \"2dee0a84-130e-413d-8294-a86bf9c82c92\") "
Oct 30 19:46:58.400488 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:58.400365    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b9fbde13-05e2-47a9-93bc-65cbd47efa6a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9fbde13-05e2-47a9-93bc-65cbd47efa6a-plugins-config-volume\") pod \"b9fbde13-05e2-47a9-93bc-65cbd47efa6a\" (UID: \"b9fbde13-05e2-47a9-93bc-65cbd47efa6a\") "
Oct 30 19:46:58.400620 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:58.400539    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2dee0a84-130e-413d-8294-a86bf9c82c92-plugins-config-volume" pod "2dee0a84-130e-413d-8294-a86bf9c82c92" (UID: "2dee0a84-130e-413d-8294-a86bf9c82c92"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:58.403171 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:58.401111    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2dee0a84-130e-413d-8294-a86bf9c82c92-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:58.403171 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:58.401342    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9fbde13-05e2-47a9-93bc-65cbd47efa6a-plugins-config-volume" pod "b9fbde13-05e2-47a9-93bc-65cbd47efa6a" (UID: "b9fbde13-05e2-47a9-93bc-65cbd47efa6a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:58.407279 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:58.407235    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2dee0a84-130e-413d-8294-a86bf9c82c92-kube-api-access-bbj8w" pod "2dee0a84-130e-413d-8294-a86bf9c82c92" (UID: "2dee0a84-130e-413d-8294-a86bf9c82c92"). InnerVolumeSpecName "kube-api-access-bbj8w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:58.407628 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:58.407603    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9fbde13-05e2-47a9-93bc-65cbd47efa6a-kube-api-access-znxqt" pod "b9fbde13-05e2-47a9-93bc-65cbd47efa6a" (UID: "b9fbde13-05e2-47a9-93bc-65cbd47efa6a"). InnerVolumeSpecName "kube-api-access-znxqt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:58.502517 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:58.502482    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-znxqt\" (UniqueName: \"kubernetes.io/projected/b9fbde13-05e2-47a9-93bc-65cbd47efa6a-kube-api-access-znxqt\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:58.502517 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:58.502514    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbj8w\" (UniqueName: \"kubernetes.io/projected/2dee0a84-130e-413d-8294-a86bf9c82c92-kube-api-access-bbj8w\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:58.502517 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:58.502523    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9fbde13-05e2-47a9-93bc-65cbd47efa6a-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:46:59.135154 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:59.135118    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-4b4c34-5bb88fdc94-s44w4"
Oct 30 19:46:59.135597 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:59.135148    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-4b4c34-5bb88fdc94-s44w4" event={"ID":"2dee0a84-130e-413d-8294-a86bf9c82c92","Type":"ContainerDied","Data":"5879e75c5572c759f99d2f47fef1e161f92b647d943474c8a6a997b6f3bc80b0"}
Oct 30 19:46:59.135597 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:59.135213    5907 scope.go:122] "RemoveContainer" containerID="a312f454db6b8f05a75ca06ed4c412547765da1bc3784654cb3ecf916aada2b5"
Oct 30 19:46:59.137183 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:59.137158    5907 generic.go:356] "Generic (PLEG): container finished" podID="b9fbde13-05e2-47a9-93bc-65cbd47efa6a" containerID="e3f6c6e636856fe207c94d3ce951e8abfe0e683aecc943afd6d2dcbf073e792d" exitCode=0
Oct 30 19:46:59.137300 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:59.137227    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b4c34-556f66b87f-d864z"
Oct 30 19:46:59.137300 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:59.137221    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b4c34-556f66b87f-d864z" event={"ID":"b9fbde13-05e2-47a9-93bc-65cbd47efa6a","Type":"ContainerDied","Data":"e3f6c6e636856fe207c94d3ce951e8abfe0e683aecc943afd6d2dcbf073e792d"}
Oct 30 19:46:59.137300 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:59.137284    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b4c34-556f66b87f-d864z" event={"ID":"b9fbde13-05e2-47a9-93bc-65cbd47efa6a","Type":"ContainerDied","Data":"3be947d2a1f79c706b2b8cc3c16207e7e5eba66ab81ae5f4cb24a6b1524e6b76"}
Oct 30 19:46:59.147441 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:59.147419    5907 scope.go:122] "RemoveContainer" containerID="e3f6c6e636856fe207c94d3ce951e8abfe0e683aecc943afd6d2dcbf073e792d"
Oct 30 19:46:59.157936 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:59.157916    5907 scope.go:122] "RemoveContainer" containerID="e3f6c6e636856fe207c94d3ce951e8abfe0e683aecc943afd6d2dcbf073e792d"
Oct 30 19:46:59.158270 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:46:59.158242    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3f6c6e636856fe207c94d3ce951e8abfe0e683aecc943afd6d2dcbf073e792d\": container with ID starting with e3f6c6e636856fe207c94d3ce951e8abfe0e683aecc943afd6d2dcbf073e792d not found: ID does not exist" containerID="e3f6c6e636856fe207c94d3ce951e8abfe0e683aecc943afd6d2dcbf073e792d"
Oct 30 19:46:59.158327 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:59.158284    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3f6c6e636856fe207c94d3ce951e8abfe0e683aecc943afd6d2dcbf073e792d"} err="failed to get container status \"e3f6c6e636856fe207c94d3ce951e8abfe0e683aecc943afd6d2dcbf073e792d\": rpc error: code = NotFound desc = could not find container \"e3f6c6e636856fe207c94d3ce951e8abfe0e683aecc943afd6d2dcbf073e792d\": container with ID starting with e3f6c6e636856fe207c94d3ce951e8abfe0e683aecc943afd6d2dcbf073e792d not found: ID does not exist"
Oct 30 19:46:59.194793 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:59.194760    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-4b4c34-5bb88fdc94-s44w4"]
Oct 30 19:46:59.202586 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:59.202554    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-4b4c34-5bb88fdc94-s44w4"]
Oct 30 19:46:59.246246 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:59.246152    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-4b4c34-556f66b87f-d864z"]
Oct 30 19:46:59.249242 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:46:59.249217    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-4b4c34-556f66b87f-d864z"]
Oct 30 19:47:00.289194 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.289156    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2dee0a84-130e-413d-8294-a86bf9c82c92" path="/var/lib/kubelet/pods/2dee0a84-130e-413d-8294-a86bf9c82c92/volumes"
Oct 30 19:47:00.289615 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.289585    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b9fbde13-05e2-47a9-93bc-65cbd47efa6a" path="/var/lib/kubelet/pods/b9fbde13-05e2-47a9-93bc-65cbd47efa6a/volumes"
Oct 30 19:47:00.758447 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.758410    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-291d35-888p7"]
Oct 30 19:47:00.759408 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.759388    5907 cpu_manager.go:414] "containerMap: removing container" podUID="9eb15025-db65-4deb-8a88-d6302ab53996" containerName="revoke-keys"
Oct 30 19:47:00.759459 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.759413    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9eb15025-db65-4deb-8a88-d6302ab53996" containerName="revoke-keys"
Oct 30 19:47:00.759459 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.759441    5907 cpu_manager.go:414] "containerMap: removing container" podUID="a033862c-f1bf-4331-b9dc-ebfa073c5caf" containerName="payload-pre-processing"
Oct 30 19:47:00.759459 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.759450    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a033862c-f1bf-4331-b9dc-ebfa073c5caf" containerName="payload-pre-processing"
Oct 30 19:47:00.759557 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.759467    5907 cpu_manager.go:414] "containerMap: removing container" podUID="b9fbde13-05e2-47a9-93bc-65cbd47efa6a" containerName="payload-pre-processing"
Oct 30 19:47:00.759557 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.759477    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b9fbde13-05e2-47a9-93bc-65cbd47efa6a" containerName="payload-pre-processing"
Oct 30 19:47:00.759557 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.759505    5907 cpu_manager.go:414] "containerMap: removing container" podUID="40e15386-d751-4b28-8124-a042609e0fd1" containerName="payload-processing"
Oct 30 19:47:00.759557 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.759514    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40e15386-d751-4b28-8124-a042609e0fd1" containerName="payload-processing"
Oct 30 19:47:00.759557 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.759551    5907 cpu_manager.go:414] "containerMap: removing container" podUID="e876e507-2b04-43fd-b788-b10d7d15748a" containerName="payload-processing"
Oct 30 19:47:00.759705 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.759559    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e876e507-2b04-43fd-b788-b10d7d15748a" containerName="payload-processing"
Oct 30 19:47:00.759705 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.759570    5907 cpu_manager.go:414] "containerMap: removing container" podUID="e3eabdcb-14c1-446c-88d9-261df5e254cb" containerName="revoke-keys"
Oct 30 19:47:00.759705 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.759578    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e3eabdcb-14c1-446c-88d9-261df5e254cb" containerName="revoke-keys"
Oct 30 19:47:00.759705 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.759594    5907 cpu_manager.go:414] "containerMap: removing container" podUID="2dee0a84-130e-413d-8294-a86bf9c82c92" containerName="payload-processing"
Oct 30 19:47:00.759705 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.759603    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2dee0a84-130e-413d-8294-a86bf9c82c92" containerName="payload-processing"
Oct 30 19:47:00.759853 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.759742    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="9eb15025-db65-4deb-8a88-d6302ab53996" containerName="revoke-keys"
Oct 30 19:47:00.759853 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.759760    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="a033862c-f1bf-4331-b9dc-ebfa073c5caf" containerName="payload-pre-processing"
Oct 30 19:47:00.759853 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.759771    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="b9fbde13-05e2-47a9-93bc-65cbd47efa6a" containerName="payload-pre-processing"
Oct 30 19:47:00.759853 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.759784    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="40e15386-d751-4b28-8124-a042609e0fd1" containerName="payload-processing"
Oct 30 19:47:00.759853 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.759796    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="e876e507-2b04-43fd-b788-b10d7d15748a" containerName="payload-processing"
Oct 30 19:47:00.759853 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.759804    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="e3eabdcb-14c1-446c-88d9-261df5e254cb" containerName="revoke-keys"
Oct 30 19:47:00.759853 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.759814    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="2dee0a84-130e-413d-8294-a86bf9c82c92" containerName="payload-processing"
Oct 30 19:47:00.765397 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.765375    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-291d35-888p7"
Oct 30 19:47:00.783705 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.783650    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-291d35-888p7"]
Oct 30 19:47:00.820358 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.820325    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:47:00.820584 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.820364    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:47:00.842937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.842899    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsrln\" (UniqueName: \"kubernetes.io/projected/ea7416ac-237e-409a-b95c-2c5d61119033-kube-api-access-gsrln\") pod \"maas-api-revoke-keys-e2e-shared-b-w1-291d35-888p7\" (UID: \"ea7416ac-237e-409a-b95c-2c5d61119033\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-291d35-888p7"
Oct 30 19:47:00.843154 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.843036    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ea7416ac-237e-409a-b95c-2c5d61119033-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w1-291d35-888p7\" (UID: \"ea7416ac-237e-409a-b95c-2c5d61119033\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-291d35-888p7"
Oct 30 19:47:00.843154 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.843067    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ea7416ac-237e-409a-b95c-2c5d61119033-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w1-291d35-888p7\" (UID: \"ea7416ac-237e-409a-b95c-2c5d61119033\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-291d35-888p7"
Oct 30 19:47:00.944357 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.944319    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ea7416ac-237e-409a-b95c-2c5d61119033-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w1-291d35-888p7\" (UID: \"ea7416ac-237e-409a-b95c-2c5d61119033\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-291d35-888p7"
Oct 30 19:47:00.944357 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.944358    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ea7416ac-237e-409a-b95c-2c5d61119033-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w1-291d35-888p7\" (UID: \"ea7416ac-237e-409a-b95c-2c5d61119033\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-291d35-888p7"
Oct 30 19:47:00.944573 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.944556    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsrln\" (UniqueName: \"kubernetes.io/projected/ea7416ac-237e-409a-b95c-2c5d61119033-kube-api-access-gsrln\") pod \"maas-api-revoke-keys-e2e-shared-b-w1-291d35-888p7\" (UID: \"ea7416ac-237e-409a-b95c-2c5d61119033\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-291d35-888p7"
Oct 30 19:47:00.945011 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.944987    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ea7416ac-237e-409a-b95c-2c5d61119033-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w1-291d35-888p7\" (UID: \"ea7416ac-237e-409a-b95c-2c5d61119033\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-291d35-888p7"
Oct 30 19:47:00.946926 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.946909    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ea7416ac-237e-409a-b95c-2c5d61119033-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w1-291d35-888p7\" (UID: \"ea7416ac-237e-409a-b95c-2c5d61119033\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-291d35-888p7"
Oct 30 19:47:00.966218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:00.966192    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsrln\" (UniqueName: \"kubernetes.io/projected/ea7416ac-237e-409a-b95c-2c5d61119033-kube-api-access-gsrln\") pod \"maas-api-revoke-keys-e2e-shared-b-w1-291d35-888p7\" (UID: \"ea7416ac-237e-409a-b95c-2c5d61119033\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-291d35-888p7"
Oct 30 19:47:01.083143 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:01.083060    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-291d35-888p7"
Oct 30 19:47:01.452016 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:01.451807    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-291d35-888p7"]
Oct 30 19:47:02.176910 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:02.176836    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-291d35-888p7" event={"ID":"ea7416ac-237e-409a-b95c-2c5d61119033","Type":"ContainerStarted","Data":"f9912a04ad55516eac512d90e632e752e2a09633801d47b22a7c149df41f19a6"}
Oct 30 19:47:02.177098 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:02.176941    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-291d35-888p7" event={"ID":"ea7416ac-237e-409a-b95c-2c5d61119033","Type":"ContainerStarted","Data":"90bf2363afbe730a506c4a7070508d23d3c3995e28c84c36fa4518bb8d402353"}
Oct 30 19:47:03.043721 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:03.043653    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-291d35-888p7" podStartSLOduration=3.043633347 podStartE2EDuration="3.043633347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:47:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:02.263331806 +0000 UTC m=+1534.106263745" watchObservedRunningTime="2026-10-30 19:47:03.043633347 +0000 UTC m=+1534.886565292"
Oct 30 19:47:03.045329 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:03.045298    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf"]
Oct 30 19:47:03.045584 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:03.045552    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf" podUID="7b021bfa-85df-4d44-8717-3c99b0ea7ae1" containerName="maas-api" containerID="cri-o://877531ad62d78646b8439bfa6b8f8b371ad1857599185356d5238a5da6f5a97d" gracePeriod=30
Oct 30 19:47:03.187597 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:03.187560    5907 generic.go:356] "Generic (PLEG): container finished" podID="ea7416ac-237e-409a-b95c-2c5d61119033" containerID="f9912a04ad55516eac512d90e632e752e2a09633801d47b22a7c149df41f19a6" exitCode=0
Oct 30 19:47:03.187800 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:03.187650    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-291d35-888p7" event={"ID":"ea7416ac-237e-409a-b95c-2c5d61119033","Type":"ContainerDied","Data":"f9912a04ad55516eac512d90e632e752e2a09633801d47b22a7c149df41f19a6"}
Oct 30 19:47:03.189516 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:03.189483    5907 generic.go:356] "Generic (PLEG): container finished" podID="7b021bfa-85df-4d44-8717-3c99b0ea7ae1" containerID="877531ad62d78646b8439bfa6b8f8b371ad1857599185356d5238a5da6f5a97d" exitCode=0
Oct 30 19:47:03.189671 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:03.189565    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf" event={"ID":"7b021bfa-85df-4d44-8717-3c99b0ea7ae1","Type":"ContainerDied","Data":"877531ad62d78646b8439bfa6b8f8b371ad1857599185356d5238a5da6f5a97d"}
Oct 30 19:47:03.509888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:03.506560    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf"
Oct 30 19:47:03.630026 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:03.629996    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7b021bfa-85df-4d44-8717-3c99b0ea7ae1-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b021bfa-85df-4d44-8717-3c99b0ea7ae1-maas-api-tls\") pod \"7b021bfa-85df-4d44-8717-3c99b0ea7ae1\" (UID: \"7b021bfa-85df-4d44-8717-3c99b0ea7ae1\") "
Oct 30 19:47:03.630218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:03.630046    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7b021bfa-85df-4d44-8717-3c99b0ea7ae1-metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b021bfa-85df-4d44-8717-3c99b0ea7ae1-metrics-certs\") pod \"7b021bfa-85df-4d44-8717-3c99b0ea7ae1\" (UID: \"7b021bfa-85df-4d44-8717-3c99b0ea7ae1\") "
Oct 30 19:47:03.630218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:03.630086    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7b021bfa-85df-4d44-8717-3c99b0ea7ae1-kube-api-access-9thwk\" (UniqueName: \"kubernetes.io/projected/7b021bfa-85df-4d44-8717-3c99b0ea7ae1-kube-api-access-9thwk\") pod \"7b021bfa-85df-4d44-8717-3c99b0ea7ae1\" (UID: \"7b021bfa-85df-4d44-8717-3c99b0ea7ae1\") "
Oct 30 19:47:03.630325 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:03.630308    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7b021bfa-85df-4d44-8717-3c99b0ea7ae1-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b021bfa-85df-4d44-8717-3c99b0ea7ae1-openshift-service-ca\") pod \"7b021bfa-85df-4d44-8717-3c99b0ea7ae1\" (UID: \"7b021bfa-85df-4d44-8717-3c99b0ea7ae1\") "
Oct 30 19:47:03.630885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:03.630739    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b021bfa-85df-4d44-8717-3c99b0ea7ae1-openshift-service-ca" pod "7b021bfa-85df-4d44-8717-3c99b0ea7ae1" (UID: "7b021bfa-85df-4d44-8717-3c99b0ea7ae1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:47:03.632366 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:03.632339    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b021bfa-85df-4d44-8717-3c99b0ea7ae1-maas-api-tls" pod "7b021bfa-85df-4d44-8717-3c99b0ea7ae1" (UID: "7b021bfa-85df-4d44-8717-3c99b0ea7ae1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:47:03.632759 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:03.632743    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b021bfa-85df-4d44-8717-3c99b0ea7ae1-metrics-certs" pod "7b021bfa-85df-4d44-8717-3c99b0ea7ae1" (UID: "7b021bfa-85df-4d44-8717-3c99b0ea7ae1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:47:03.632829 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:03.632814    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b021bfa-85df-4d44-8717-3c99b0ea7ae1-kube-api-access-9thwk" pod "7b021bfa-85df-4d44-8717-3c99b0ea7ae1" (UID: "7b021bfa-85df-4d44-8717-3c99b0ea7ae1"). InnerVolumeSpecName "kube-api-access-9thwk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:03.732481 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:03.732391    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b021bfa-85df-4d44-8717-3c99b0ea7ae1-maas-api-tls\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:03.732481 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:03.732429    5907 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b021bfa-85df-4d44-8717-3c99b0ea7ae1-metrics-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:03.732481 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:03.732440    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9thwk\" (UniqueName: \"kubernetes.io/projected/7b021bfa-85df-4d44-8717-3c99b0ea7ae1-kube-api-access-9thwk\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:03.732481 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:03.732450    5907 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b021bfa-85df-4d44-8717-3c99b0ea7ae1-openshift-service-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:03.865428 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:03.865394    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9de441-4zkxr"]
Oct 30 19:47:03.869147 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:03.869118    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9de441-4zkxr"]
Oct 30 19:47:04.198846 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:04.198818    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf"
Oct 30 19:47:04.199360 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:04.198848    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf" event={"ID":"7b021bfa-85df-4d44-8717-3c99b0ea7ae1","Type":"ContainerDied","Data":"3d18d533681716acfe4561861735cb3260527839d95f13dbc58397ce4b1fbb62"}
Oct 30 19:47:04.199360 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:04.198938    5907 scope.go:122] "RemoveContainer" containerID="877531ad62d78646b8439bfa6b8f8b371ad1857599185356d5238a5da6f5a97d"
Oct 30 19:47:04.265630 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:04.265563    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf"]
Oct 30 19:47:04.270627 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:04.270597    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9de441-c7557bbd5-z5mkf"]
Oct 30 19:47:04.291715 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:04.291606    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4179be3f-8227-4cab-aa4a-62825fa5fe03" path="/var/lib/kubelet/pods/4179be3f-8227-4cab-aa4a-62825fa5fe03/volumes"
Oct 30 19:47:04.292265 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:04.292243    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7b021bfa-85df-4d44-8717-3c99b0ea7ae1" path="/var/lib/kubelet/pods/7b021bfa-85df-4d44-8717-3c99b0ea7ae1/volumes"
Oct 30 19:47:04.356698 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:04.356673    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-291d35-888p7"
Oct 30 19:47:04.471740 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:04.471702    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ea7416ac-237e-409a-b95c-2c5d61119033-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ea7416ac-237e-409a-b95c-2c5d61119033-maas-api-xks-ca\") pod \"ea7416ac-237e-409a-b95c-2c5d61119033\" (UID: \"ea7416ac-237e-409a-b95c-2c5d61119033\") "
Oct 30 19:47:04.471939 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:04.471898    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ea7416ac-237e-409a-b95c-2c5d61119033-kube-api-access-gsrln\" (UniqueName: \"kubernetes.io/projected/ea7416ac-237e-409a-b95c-2c5d61119033-kube-api-access-gsrln\") pod \"ea7416ac-237e-409a-b95c-2c5d61119033\" (UID: \"ea7416ac-237e-409a-b95c-2c5d61119033\") "
Oct 30 19:47:04.472040 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:04.472021    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ea7416ac-237e-409a-b95c-2c5d61119033-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ea7416ac-237e-409a-b95c-2c5d61119033-maas-api-openshift-ca\") pod \"ea7416ac-237e-409a-b95c-2c5d61119033\" (UID: \"ea7416ac-237e-409a-b95c-2c5d61119033\") "
Oct 30 19:47:04.472396 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:04.472368    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea7416ac-237e-409a-b95c-2c5d61119033-maas-api-openshift-ca" pod "ea7416ac-237e-409a-b95c-2c5d61119033" (UID: "ea7416ac-237e-409a-b95c-2c5d61119033"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:47:04.472638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:04.472610    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ea7416ac-237e-409a-b95c-2c5d61119033-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:04.474109 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:04.474089    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea7416ac-237e-409a-b95c-2c5d61119033-kube-api-access-gsrln" pod "ea7416ac-237e-409a-b95c-2c5d61119033" (UID: "ea7416ac-237e-409a-b95c-2c5d61119033"). InnerVolumeSpecName "kube-api-access-gsrln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:04.474192 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:04.474175    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea7416ac-237e-409a-b95c-2c5d61119033-maas-api-xks-ca" pod "ea7416ac-237e-409a-b95c-2c5d61119033" (UID: "ea7416ac-237e-409a-b95c-2c5d61119033"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:47:04.574136 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:04.574101    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gsrln\" (UniqueName: \"kubernetes.io/projected/ea7416ac-237e-409a-b95c-2c5d61119033-kube-api-access-gsrln\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:04.574136 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:04.574139    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ea7416ac-237e-409a-b95c-2c5d61119033-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:05.235885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:05.232947    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-291d35-888p7" event={"ID":"ea7416ac-237e-409a-b95c-2c5d61119033","Type":"ContainerDied","Data":"90bf2363afbe730a506c4a7070508d23d3c3995e28c84c36fa4518bb8d402353"}
Oct 30 19:47:05.235885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:05.233017    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="90bf2363afbe730a506c4a7070508d23d3c3995e28c84c36fa4518bb8d402353"
Oct 30 19:47:05.235885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:05.233094    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-291d35-888p7"
Oct 30 19:47:06.671318 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:06.671284    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4b4c34-jccpb"]
Oct 30 19:47:06.683998 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:06.683968    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-4b4c34-jccpb"]
Oct 30 19:47:08.388695 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:08.388609    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4a53e23a-fa9d-43f9-a651-baa25612b463" path="/var/lib/kubelet/pods/4a53e23a-fa9d-43f9-a651-baa25612b463/volumes"
Oct 30 19:47:10.043591 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:10.043556    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-43eebade-tlhqf"]
Oct 30 19:47:10.056557 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:10.056532    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-43eebade-tlhqf"]
Oct 30 19:47:10.290342 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:10.290306    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a37f0676-70db-40df-a028-8edb736a903c" path="/var/lib/kubelet/pods/a37f0676-70db-40df-a028-8edb736a903c/volumes"
Oct 30 19:47:11.239756 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:11.239720    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-291d35-7fd55c44f6-fc8sp"]
Oct 30 19:47:11.284843 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:11.284810    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-291d35-5d99595567-6sjwx"]
Oct 30 19:47:13.813397 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:13.813354    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"]
Oct 30 19:47:13.813914 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:13.813646    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599" podUID="0da36910-7634-4556-b95b-451340c2a584" containerName="istio-proxy" containerID="cri-o://6b7a2f4c29214fc5835150fb7c7903225270f1c9bedfaf731cebcc6551b920c6" gracePeriod=30
Oct 30 19:47:14.567128 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:14.567087    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f913c6-8g4fc"]
Oct 30 19:47:14.567750 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:14.567730    5907 cpu_manager.go:414] "containerMap: removing container" podUID="ea7416ac-237e-409a-b95c-2c5d61119033" containerName="revoke-keys"
Oct 30 19:47:14.567890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:14.567752    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ea7416ac-237e-409a-b95c-2c5d61119033" containerName="revoke-keys"
Oct 30 19:47:14.567890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:14.567835    5907 cpu_manager.go:414] "containerMap: removing container" podUID="7b021bfa-85df-4d44-8717-3c99b0ea7ae1" containerName="maas-api"
Oct 30 19:47:14.567890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:14.567844    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7b021bfa-85df-4d44-8717-3c99b0ea7ae1" containerName="maas-api"
Oct 30 19:47:14.568078 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:14.568007    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="7b021bfa-85df-4d44-8717-3c99b0ea7ae1" containerName="maas-api"
Oct 30 19:47:14.568078 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:14.568026    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="ea7416ac-237e-409a-b95c-2c5d61119033" containerName="revoke-keys"
Oct 30 19:47:14.573060 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:14.573038    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f913c6-8g4fc"
Oct 30 19:47:14.583903 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:14.583879    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c0fc0b73-412c-46de-bac5-69f35c8c229b-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-f913c6-8g4fc\" (UID: \"c0fc0b73-412c-46de-bac5-69f35c8c229b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f913c6-8g4fc"
Oct 30 19:47:14.584026 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:14.583924    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c0fc0b73-412c-46de-bac5-69f35c8c229b-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-f913c6-8g4fc\" (UID: \"c0fc0b73-412c-46de-bac5-69f35c8c229b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f913c6-8g4fc"
Oct 30 19:47:14.584026 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:14.584001    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6gqb\" (UniqueName: \"kubernetes.io/projected/c0fc0b73-412c-46de-bac5-69f35c8c229b-kube-api-access-f6gqb\") pod \"maas-api-revoke-keys-e2e-worker-w4-f913c6-8g4fc\" (UID: \"c0fc0b73-412c-46de-bac5-69f35c8c229b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f913c6-8g4fc"
Oct 30 19:47:14.594751 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:14.594702    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f913c6-8g4fc"]
Oct 30 19:47:14.599117 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:14.598993    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:47:14.604088 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:14.603995    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:47:14.685590 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:14.685555    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c0fc0b73-412c-46de-bac5-69f35c8c229b-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-f913c6-8g4fc\" (UID: \"c0fc0b73-412c-46de-bac5-69f35c8c229b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f913c6-8g4fc"
Oct 30 19:47:14.685773 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:14.685617    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c0fc0b73-412c-46de-bac5-69f35c8c229b-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-f913c6-8g4fc\" (UID: \"c0fc0b73-412c-46de-bac5-69f35c8c229b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f913c6-8g4fc"
Oct 30 19:47:14.685773 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:14.685702    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6gqb\" (UniqueName: \"kubernetes.io/projected/c0fc0b73-412c-46de-bac5-69f35c8c229b-kube-api-access-f6gqb\") pod \"maas-api-revoke-keys-e2e-worker-w4-f913c6-8g4fc\" (UID: \"c0fc0b73-412c-46de-bac5-69f35c8c229b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f913c6-8g4fc"
Oct 30 19:47:14.686417 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:14.686396    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c0fc0b73-412c-46de-bac5-69f35c8c229b-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-f913c6-8g4fc\" (UID: \"c0fc0b73-412c-46de-bac5-69f35c8c229b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f913c6-8g4fc"
Oct 30 19:47:14.688211 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:14.688185    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c0fc0b73-412c-46de-bac5-69f35c8c229b-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-f913c6-8g4fc\" (UID: \"c0fc0b73-412c-46de-bac5-69f35c8c229b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f913c6-8g4fc"
Oct 30 19:47:14.708492 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:14.708462    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6gqb\" (UniqueName: \"kubernetes.io/projected/c0fc0b73-412c-46de-bac5-69f35c8c229b-kube-api-access-f6gqb\") pod \"maas-api-revoke-keys-e2e-worker-w4-f913c6-8g4fc\" (UID: \"c0fc0b73-412c-46de-bac5-69f35c8c229b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f913c6-8g4fc"
Oct 30 19:47:14.887705 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:14.887668    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f913c6-8g4fc"
Oct 30 19:47:15.199474 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:15.199444    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f913c6-8g4fc"]
Oct 30 19:47:15.201688 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:47:15.201654    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0fc0b73_412c_46de_bac5_69f35c8c229b.slice/crio-5008f22ad536cf233100ffbeb0ed2c9fe278e7f91355feb396097aca9517713d WatchSource:0}: Error finding container 5008f22ad536cf233100ffbeb0ed2c9fe278e7f91355feb396097aca9517713d: Status 404 returned error can't find the container with id 5008f22ad536cf233100ffbeb0ed2c9fe278e7f91355feb396097aca9517713d
Oct 30 19:47:15.454692 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:15.454614    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f913c6-8g4fc" event={"ID":"c0fc0b73-412c-46de-bac5-69f35c8c229b","Type":"ContainerStarted","Data":"5008f22ad536cf233100ffbeb0ed2c9fe278e7f91355feb396097aca9517713d"}
Oct 30 19:47:16.241012 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:16.240938    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-291d35-7fd55c44f6-fc8sp" podUID="79819aff-c2ad-43e2-9155-aa91c9c38aa7" containerName="payload-processing" containerID="cri-o://d73ac53c62eb070bea0bb17c082707f54e328208ed51295b60beece727360530" gracePeriod=25
Oct 30 19:47:16.286090 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:16.286045    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-291d35-5d99595567-6sjwx" podUID="f67cd235-0177-4e87-b233-1e48dd296e3b" containerName="payload-pre-processing" containerID="cri-o://538d929616656ff4f5b6c8ce2500a8ef7da03a37efdc82fb1dc5b2976269f2af" gracePeriod=25
Oct 30 19:47:16.470905 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:16.468259    5907 generic.go:356] "Generic (PLEG): container finished" podID="f67cd235-0177-4e87-b233-1e48dd296e3b" containerID="538d929616656ff4f5b6c8ce2500a8ef7da03a37efdc82fb1dc5b2976269f2af" exitCode=0
Oct 30 19:47:16.470905 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:16.468330    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-291d35-5d99595567-6sjwx" event={"ID":"f67cd235-0177-4e87-b233-1e48dd296e3b","Type":"ContainerDied","Data":"538d929616656ff4f5b6c8ce2500a8ef7da03a37efdc82fb1dc5b2976269f2af"}
Oct 30 19:47:16.470905 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:16.470536    5907 generic.go:356] "Generic (PLEG): container finished" podID="c0fc0b73-412c-46de-bac5-69f35c8c229b" containerID="59d996e025b64708a0bd38f4f6a8ffeabbc15a6609d64a590e584a1ee57130de" exitCode=0
Oct 30 19:47:16.471291 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:16.471265    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f913c6-8g4fc" event={"ID":"c0fc0b73-412c-46de-bac5-69f35c8c229b","Type":"ContainerDied","Data":"59d996e025b64708a0bd38f4f6a8ffeabbc15a6609d64a590e584a1ee57130de"}
Oct 30 19:47:16.473400 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:16.473368    5907 generic.go:356] "Generic (PLEG): container finished" podID="79819aff-c2ad-43e2-9155-aa91c9c38aa7" containerID="d73ac53c62eb070bea0bb17c082707f54e328208ed51295b60beece727360530" exitCode=0
Oct 30 19:47:16.473575 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:16.473440    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-291d35-7fd55c44f6-fc8sp" event={"ID":"79819aff-c2ad-43e2-9155-aa91c9c38aa7","Type":"ContainerDied","Data":"d73ac53c62eb070bea0bb17c082707f54e328208ed51295b60beece727360530"}
Oct 30 19:47:16.609006 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:16.608979    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-291d35-7fd55c44f6-fc8sp"
Oct 30 19:47:16.620873 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:16.620838    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-291d35-5d99595567-6sjwx"
Oct 30 19:47:16.625851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:16.625830    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/79819aff-c2ad-43e2-9155-aa91c9c38aa7-kube-api-access-4658q\" (UniqueName: \"kubernetes.io/projected/79819aff-c2ad-43e2-9155-aa91c9c38aa7-kube-api-access-4658q\") pod \"79819aff-c2ad-43e2-9155-aa91c9c38aa7\" (UID: \"79819aff-c2ad-43e2-9155-aa91c9c38aa7\") "
Oct 30 19:47:16.626007 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:16.625880    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/79819aff-c2ad-43e2-9155-aa91c9c38aa7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79819aff-c2ad-43e2-9155-aa91c9c38aa7-plugins-config-volume\") pod \"79819aff-c2ad-43e2-9155-aa91c9c38aa7\" (UID: \"79819aff-c2ad-43e2-9155-aa91c9c38aa7\") "
Oct 30 19:47:16.626264 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:16.626243    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79819aff-c2ad-43e2-9155-aa91c9c38aa7-plugins-config-volume" pod "79819aff-c2ad-43e2-9155-aa91c9c38aa7" (UID: "79819aff-c2ad-43e2-9155-aa91c9c38aa7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:47:16.626410 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:16.626395    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79819aff-c2ad-43e2-9155-aa91c9c38aa7-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:16.628093 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:16.628074    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79819aff-c2ad-43e2-9155-aa91c9c38aa7-kube-api-access-4658q" pod "79819aff-c2ad-43e2-9155-aa91c9c38aa7" (UID: "79819aff-c2ad-43e2-9155-aa91c9c38aa7"). InnerVolumeSpecName "kube-api-access-4658q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:16.727296 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:16.727266    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f67cd235-0177-4e87-b233-1e48dd296e3b-kube-api-access-zb868\" (UniqueName: \"kubernetes.io/projected/f67cd235-0177-4e87-b233-1e48dd296e3b-kube-api-access-zb868\") pod \"f67cd235-0177-4e87-b233-1e48dd296e3b\" (UID: \"f67cd235-0177-4e87-b233-1e48dd296e3b\") "
Oct 30 19:47:16.727491 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:16.727421    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f67cd235-0177-4e87-b233-1e48dd296e3b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f67cd235-0177-4e87-b233-1e48dd296e3b-plugins-config-volume\") pod \"f67cd235-0177-4e87-b233-1e48dd296e3b\" (UID: \"f67cd235-0177-4e87-b233-1e48dd296e3b\") "
Oct 30 19:47:16.727812 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:16.727785    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f67cd235-0177-4e87-b233-1e48dd296e3b-plugins-config-volume" pod "f67cd235-0177-4e87-b233-1e48dd296e3b" (UID: "f67cd235-0177-4e87-b233-1e48dd296e3b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:47:16.728025 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:16.728008    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f67cd235-0177-4e87-b233-1e48dd296e3b-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:16.728091 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:16.728031    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4658q\" (UniqueName: \"kubernetes.io/projected/79819aff-c2ad-43e2-9155-aa91c9c38aa7-kube-api-access-4658q\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:16.729695 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:16.729669    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f67cd235-0177-4e87-b233-1e48dd296e3b-kube-api-access-zb868" pod "f67cd235-0177-4e87-b233-1e48dd296e3b" (UID: "f67cd235-0177-4e87-b233-1e48dd296e3b"). InnerVolumeSpecName "kube-api-access-zb868". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:16.829449 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:16.829368    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zb868\" (UniqueName: \"kubernetes.io/projected/f67cd235-0177-4e87-b233-1e48dd296e3b-kube-api-access-zb868\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:17.484345 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:17.484320    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-291d35-5d99595567-6sjwx"
Oct 30 19:47:17.484814 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:17.484342    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-291d35-5d99595567-6sjwx" event={"ID":"f67cd235-0177-4e87-b233-1e48dd296e3b","Type":"ContainerDied","Data":"e4efae4c8c75ecb220a785758d25a546c22e73cb339fce961160334b9ff67b33"}
Oct 30 19:47:17.484814 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:17.484415    5907 scope.go:122] "RemoveContainer" containerID="538d929616656ff4f5b6c8ce2500a8ef7da03a37efdc82fb1dc5b2976269f2af"
Oct 30 19:47:17.489097 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:17.489064    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-291d35-7fd55c44f6-fc8sp" event={"ID":"79819aff-c2ad-43e2-9155-aa91c9c38aa7","Type":"ContainerDied","Data":"98c78f6243b608422ded8e29caae7e96a443138b627a6e81063e9c0ac61dfb86"}
Oct 30 19:47:17.489226 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:17.489073    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-291d35-7fd55c44f6-fc8sp"
Oct 30 19:47:17.501342 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:17.501305    5907 scope.go:122] "RemoveContainer" containerID="d73ac53c62eb070bea0bb17c082707f54e328208ed51295b60beece727360530"
Oct 30 19:47:17.510139 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:47:17.509406    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:47:17.550407 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:17.550370    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-291d35-5d99595567-6sjwx"]
Oct 30 19:47:17.569609 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:17.569521    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-291d35-5d99595567-6sjwx"]
Oct 30 19:47:17.607978 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:17.607945    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-291d35-7fd55c44f6-fc8sp"]
Oct 30 19:47:17.615042 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:17.615015    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-291d35-7fd55c44f6-fc8sp"]
Oct 30 19:47:17.655581 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:17.655555    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f913c6-8g4fc"
Oct 30 19:47:17.748552 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:17.748447    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c0fc0b73-412c-46de-bac5-69f35c8c229b-kube-api-access-f6gqb\" (UniqueName: \"kubernetes.io/projected/c0fc0b73-412c-46de-bac5-69f35c8c229b-kube-api-access-f6gqb\") pod \"c0fc0b73-412c-46de-bac5-69f35c8c229b\" (UID: \"c0fc0b73-412c-46de-bac5-69f35c8c229b\") "
Oct 30 19:47:17.748739 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:17.748630    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c0fc0b73-412c-46de-bac5-69f35c8c229b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c0fc0b73-412c-46de-bac5-69f35c8c229b-maas-api-xks-ca\") pod \"c0fc0b73-412c-46de-bac5-69f35c8c229b\" (UID: \"c0fc0b73-412c-46de-bac5-69f35c8c229b\") "
Oct 30 19:47:17.748846 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:17.748822    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c0fc0b73-412c-46de-bac5-69f35c8c229b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c0fc0b73-412c-46de-bac5-69f35c8c229b-maas-api-openshift-ca\") pod \"c0fc0b73-412c-46de-bac5-69f35c8c229b\" (UID: \"c0fc0b73-412c-46de-bac5-69f35c8c229b\") "
Oct 30 19:47:17.749324 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:17.749294    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0fc0b73-412c-46de-bac5-69f35c8c229b-maas-api-openshift-ca" pod "c0fc0b73-412c-46de-bac5-69f35c8c229b" (UID: "c0fc0b73-412c-46de-bac5-69f35c8c229b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:47:17.749927 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:17.749904    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c0fc0b73-412c-46de-bac5-69f35c8c229b-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:17.751403 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:17.751381    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0fc0b73-412c-46de-bac5-69f35c8c229b-maas-api-xks-ca" pod "c0fc0b73-412c-46de-bac5-69f35c8c229b" (UID: "c0fc0b73-412c-46de-bac5-69f35c8c229b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:47:17.751490 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:17.751440    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0fc0b73-412c-46de-bac5-69f35c8c229b-kube-api-access-f6gqb" pod "c0fc0b73-412c-46de-bac5-69f35c8c229b" (UID: "c0fc0b73-412c-46de-bac5-69f35c8c229b"). InnerVolumeSpecName "kube-api-access-f6gqb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:17.851620 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:17.851590    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6gqb\" (UniqueName: \"kubernetes.io/projected/c0fc0b73-412c-46de-bac5-69f35c8c229b-kube-api-access-f6gqb\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:17.851620 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:17.851619    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c0fc0b73-412c-46de-bac5-69f35c8c229b-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:18.288783 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:18.288750    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="79819aff-c2ad-43e2-9155-aa91c9c38aa7" path="/var/lib/kubelet/pods/79819aff-c2ad-43e2-9155-aa91c9c38aa7/volumes"
Oct 30 19:47:18.289222 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:18.289207    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f67cd235-0177-4e87-b233-1e48dd296e3b" path="/var/lib/kubelet/pods/f67cd235-0177-4e87-b233-1e48dd296e3b/volumes"
Oct 30 19:47:18.499335 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:18.499293    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f913c6-8g4fc" event={"ID":"c0fc0b73-412c-46de-bac5-69f35c8c229b","Type":"ContainerDied","Data":"5008f22ad536cf233100ffbeb0ed2c9fe278e7f91355feb396097aca9517713d"}
Oct 30 19:47:18.499814 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:18.499359    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5008f22ad536cf233100ffbeb0ed2c9fe278e7f91355feb396097aca9517713d"
Oct 30 19:47:18.499814 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:18.499315    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f913c6-8g4fc"
Oct 30 19:47:18.693732 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:18.693696    5907 patch_prober.go:29] interesting pod/e2e-api-b-9de441-openshift-default-655b8b8797-ps599 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:47:18.693929 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:18.693751    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599" podUID="0da36910-7634-4556-b95b-451340c2a584" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:47:19.081807 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.081779    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:47:19.168818 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.168783    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0da36910-7634-4556-b95b-451340c2a584-istio-token\" (UniqueName: \"kubernetes.io/projected/0da36910-7634-4556-b95b-451340c2a584-istio-token\") pod \"0da36910-7634-4556-b95b-451340c2a584\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") "
Oct 30 19:47:19.169005 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.168898    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0da36910-7634-4556-b95b-451340c2a584-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da36910-7634-4556-b95b-451340c2a584-workload-socket\") pod \"0da36910-7634-4556-b95b-451340c2a584\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") "
Oct 30 19:47:19.169005 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.168956    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0da36910-7634-4556-b95b-451340c2a584-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da36910-7634-4556-b95b-451340c2a584-credential-socket\") pod \"0da36910-7634-4556-b95b-451340c2a584\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") "
Oct 30 19:47:19.169144 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.169044    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0da36910-7634-4556-b95b-451340c2a584-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0da36910-7634-4556-b95b-451340c2a584-workload-certs\") pod \"0da36910-7634-4556-b95b-451340c2a584\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") "
Oct 30 19:47:19.169144 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.169082    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0da36910-7634-4556-b95b-451340c2a584-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0da36910-7634-4556-b95b-451340c2a584-istio-envoy\") pod \"0da36910-7634-4556-b95b-451340c2a584\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") "
Oct 30 19:47:19.169144 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.169117    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0da36910-7634-4556-b95b-451340c2a584-kube-api-access-tmh6r\" (UniqueName: \"kubernetes.io/projected/0da36910-7634-4556-b95b-451340c2a584-kube-api-access-tmh6r\") pod \"0da36910-7634-4556-b95b-451340c2a584\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") "
Oct 30 19:47:19.169323 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.169159    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0da36910-7634-4556-b95b-451340c2a584-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0da36910-7634-4556-b95b-451340c2a584-istio-podinfo\") pod \"0da36910-7634-4556-b95b-451340c2a584\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") "
Oct 30 19:47:19.169323 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.169242    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0da36910-7634-4556-b95b-451340c2a584-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0da36910-7634-4556-b95b-451340c2a584-istiod-ca-cert\") pod \"0da36910-7634-4556-b95b-451340c2a584\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") "
Oct 30 19:47:19.169323 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.169264    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0da36910-7634-4556-b95b-451340c2a584-credential-socket" pod "0da36910-7634-4556-b95b-451340c2a584" (UID: "0da36910-7634-4556-b95b-451340c2a584"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:19.169323 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.169305    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0da36910-7634-4556-b95b-451340c2a584-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0da36910-7634-4556-b95b-451340c2a584-istio-data\") pod \"0da36910-7634-4556-b95b-451340c2a584\" (UID: \"0da36910-7634-4556-b95b-451340c2a584\") "
Oct 30 19:47:19.169628 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.169563    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0da36910-7634-4556-b95b-451340c2a584-workload-certs" pod "0da36910-7634-4556-b95b-451340c2a584" (UID: "0da36910-7634-4556-b95b-451340c2a584"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:19.169628 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.169617    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0da36910-7634-4556-b95b-451340c2a584-workload-socket" pod "0da36910-7634-4556-b95b-451340c2a584" (UID: "0da36910-7634-4556-b95b-451340c2a584"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:19.169950 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.169782    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0da36910-7634-4556-b95b-451340c2a584-istio-data" pod "0da36910-7634-4556-b95b-451340c2a584" (UID: "0da36910-7634-4556-b95b-451340c2a584"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:19.170019 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.169950    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0da36910-7634-4556-b95b-451340c2a584-istiod-ca-cert" pod "0da36910-7634-4556-b95b-451340c2a584" (UID: "0da36910-7634-4556-b95b-451340c2a584"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:47:19.170143 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.170014    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0da36910-7634-4556-b95b-451340c2a584-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:19.170143 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.170035    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0da36910-7634-4556-b95b-451340c2a584-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:19.170143 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.170048    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da36910-7634-4556-b95b-451340c2a584-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:19.170143 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.170062    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da36910-7634-4556-b95b-451340c2a584-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:19.171478 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.171453    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0da36910-7634-4556-b95b-451340c2a584-istio-token" pod "0da36910-7634-4556-b95b-451340c2a584" (UID: "0da36910-7634-4556-b95b-451340c2a584"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:19.171816 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.171795    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0da36910-7634-4556-b95b-451340c2a584-kube-api-access-tmh6r" pod "0da36910-7634-4556-b95b-451340c2a584" (UID: "0da36910-7634-4556-b95b-451340c2a584"). InnerVolumeSpecName "kube-api-access-tmh6r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:19.172127 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.172106    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0da36910-7634-4556-b95b-451340c2a584-istio-podinfo" pod "0da36910-7634-4556-b95b-451340c2a584" (UID: "0da36910-7634-4556-b95b-451340c2a584"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:47:19.172272 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.172253    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0da36910-7634-4556-b95b-451340c2a584-istio-envoy" pod "0da36910-7634-4556-b95b-451340c2a584" (UID: "0da36910-7634-4556-b95b-451340c2a584"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:19.244216 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.244142    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"]
Oct 30 19:47:19.271450 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.271423    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0da36910-7634-4556-b95b-451340c2a584-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:19.271450 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.271450    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmh6r\" (UniqueName: \"kubernetes.io/projected/0da36910-7634-4556-b95b-451340c2a584-kube-api-access-tmh6r\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:19.271623 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.271460    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0da36910-7634-4556-b95b-451340c2a584-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:19.271623 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.271468    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0da36910-7634-4556-b95b-451340c2a584-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:19.271623 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.271476    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0da36910-7634-4556-b95b-451340c2a584-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:19.510178 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.510071    5907 generic.go:356] "Generic (PLEG): container finished" podID="0da36910-7634-4556-b95b-451340c2a584" containerID="6b7a2f4c29214fc5835150fb7c7903225270f1c9bedfaf731cebcc6551b920c6" exitCode=0
Oct 30 19:47:19.510178 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.510143    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"
Oct 30 19:47:19.510178 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.510167    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599" event={"ID":"0da36910-7634-4556-b95b-451340c2a584","Type":"ContainerDied","Data":"6b7a2f4c29214fc5835150fb7c7903225270f1c9bedfaf731cebcc6551b920c6"}
Oct 30 19:47:19.510800 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.510225    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599" event={"ID":"0da36910-7634-4556-b95b-451340c2a584","Type":"ContainerDied","Data":"f3e7ef349e88b49a99f704126acda9526fb27764948df2f8c137a6c7016307b1"}
Oct 30 19:47:19.510800 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.510251    5907 scope.go:122] "RemoveContainer" containerID="6b7a2f4c29214fc5835150fb7c7903225270f1c9bedfaf731cebcc6551b920c6"
Oct 30 19:47:19.523368 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.523351    5907 scope.go:122] "RemoveContainer" containerID="32588ff3a44c0194908475273296014ab5cba4c031e6f156a04a7bda33d45605"
Oct 30 19:47:19.534016 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.533997    5907 scope.go:122] "RemoveContainer" containerID="6b7a2f4c29214fc5835150fb7c7903225270f1c9bedfaf731cebcc6551b920c6"
Oct 30 19:47:19.534319 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:47:19.534294    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b7a2f4c29214fc5835150fb7c7903225270f1c9bedfaf731cebcc6551b920c6\": container with ID starting with 6b7a2f4c29214fc5835150fb7c7903225270f1c9bedfaf731cebcc6551b920c6 not found: ID does not exist" containerID="6b7a2f4c29214fc5835150fb7c7903225270f1c9bedfaf731cebcc6551b920c6"
Oct 30 19:47:19.534392 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.534330    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b7a2f4c29214fc5835150fb7c7903225270f1c9bedfaf731cebcc6551b920c6"} err="failed to get container status \"6b7a2f4c29214fc5835150fb7c7903225270f1c9bedfaf731cebcc6551b920c6\": rpc error: code = NotFound desc = could not find container \"6b7a2f4c29214fc5835150fb7c7903225270f1c9bedfaf731cebcc6551b920c6\": container with ID starting with 6b7a2f4c29214fc5835150fb7c7903225270f1c9bedfaf731cebcc6551b920c6 not found: ID does not exist"
Oct 30 19:47:19.534392 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.534349    5907 scope.go:122] "RemoveContainer" containerID="32588ff3a44c0194908475273296014ab5cba4c031e6f156a04a7bda33d45605"
Oct 30 19:47:19.534644 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:47:19.534615    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32588ff3a44c0194908475273296014ab5cba4c031e6f156a04a7bda33d45605\": container with ID starting with 32588ff3a44c0194908475273296014ab5cba4c031e6f156a04a7bda33d45605 not found: ID does not exist" containerID="32588ff3a44c0194908475273296014ab5cba4c031e6f156a04a7bda33d45605"
Oct 30 19:47:19.534707 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.534653    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32588ff3a44c0194908475273296014ab5cba4c031e6f156a04a7bda33d45605"} err="failed to get container status \"32588ff3a44c0194908475273296014ab5cba4c031e6f156a04a7bda33d45605\": rpc error: code = NotFound desc = could not find container \"32588ff3a44c0194908475273296014ab5cba4c031e6f156a04a7bda33d45605\": container with ID starting with 32588ff3a44c0194908475273296014ab5cba4c031e6f156a04a7bda33d45605 not found: ID does not exist"
Oct 30 19:47:19.569077 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.569050    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"]
Oct 30 19:47:19.581618 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:19.581587    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-9de441-openshift-default-655b8b8797-ps599"]
Oct 30 19:47:20.289957 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:20.289917    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0da36910-7634-4556-b95b-451340c2a584" path="/var/lib/kubelet/pods/0da36910-7634-4556-b95b-451340c2a584/volumes"
Oct 30 19:47:21.734807 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:21.734773    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-291d35-888p7"]
Oct 30 19:47:21.747852 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:21.747824    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-291d35-888p7"]
Oct 30 19:47:22.288434 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:22.288396    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ea7416ac-237e-409a-b95c-2c5d61119033" path="/var/lib/kubelet/pods/ea7416ac-237e-409a-b95c-2c5d61119033/volumes"
Oct 30 19:47:24.642998 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:24.642957    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:47:27.405058 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:27.405026    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"]
Oct 30 19:47:27.405579 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:27.405336    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8" podUID="0435effb-f3d6-450f-beed-937cec7badb0" containerName="istio-proxy" containerID="cri-o://4be7617af316401736a1464fcac5393879ac9028143df75d2beae64a22f2660d" gracePeriod=30
Oct 30 19:47:29.542979 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:29.542940    5907 patch_prober.go:29] interesting pod/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:47:29.543390 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:29.543001    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8" podUID="0435effb-f3d6-450f-beed-937cec7badb0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:47:30.668186 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:30.668149    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"]
Oct 30 19:47:30.668690 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:30.668400    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd" podUID="dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe" containerName="istio-proxy" containerID="cri-o://77b47da55d91f085d97dca9028eac1585e92bd7f70c3e1fe58186110e915098d" gracePeriod=30
Oct 30 19:47:31.459173 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:31.459131    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-d797d2-cplms"]
Oct 30 19:47:31.459769 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:31.459748    5907 cpu_manager.go:414] "containerMap: removing container" podUID="0da36910-7634-4556-b95b-451340c2a584" containerName="istio-proxy"
Oct 30 19:47:31.459910 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:31.459779    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0da36910-7634-4556-b95b-451340c2a584" containerName="istio-proxy"
Oct 30 19:47:31.459910 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:31.459795    5907 cpu_manager.go:414] "containerMap: removing container" podUID="f67cd235-0177-4e87-b233-1e48dd296e3b" containerName="payload-pre-processing"
Oct 30 19:47:31.459910 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:31.459802    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f67cd235-0177-4e87-b233-1e48dd296e3b" containerName="payload-pre-processing"
Oct 30 19:47:31.459910 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:31.459824    5907 cpu_manager.go:414] "containerMap: removing container" podUID="79819aff-c2ad-43e2-9155-aa91c9c38aa7" containerName="payload-processing"
Oct 30 19:47:31.459910 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:31.459831    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="79819aff-c2ad-43e2-9155-aa91c9c38aa7" containerName="payload-processing"
Oct 30 19:47:31.459910 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:31.459853    5907 cpu_manager.go:414] "containerMap: removing container" podUID="c0fc0b73-412c-46de-bac5-69f35c8c229b" containerName="revoke-keys"
Oct 30 19:47:31.459910 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:31.459876    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c0fc0b73-412c-46de-bac5-69f35c8c229b" containerName="revoke-keys"
Oct 30 19:47:31.460281 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:31.460067    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="0da36910-7634-4556-b95b-451340c2a584" containerName="istio-proxy"
Oct 30 19:47:31.460281 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:31.460083    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="f67cd235-0177-4e87-b233-1e48dd296e3b" containerName="payload-pre-processing"
Oct 30 19:47:31.460281 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:31.460097    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="79819aff-c2ad-43e2-9155-aa91c9c38aa7" containerName="payload-processing"
Oct 30 19:47:31.460281 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:31.460109    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="c0fc0b73-412c-46de-bac5-69f35c8c229b" containerName="revoke-keys"
Oct 30 19:47:31.465410 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:31.465387    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-d797d2-cplms"
Oct 30 19:47:31.484611 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:31.484577    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-d797d2-cplms"]
Oct 30 19:47:31.494390 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:31.494364    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:47:31.494520 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:31.494461    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:47:31.503055 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:31.503030    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c960481b-979f-46ab-9edb-78ea467367eb-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-d797d2-cplms\" (UID: \"c960481b-979f-46ab-9edb-78ea467367eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-d797d2-cplms"
Oct 30 19:47:31.503267 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:31.503239    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d56nv\" (UniqueName: \"kubernetes.io/projected/c960481b-979f-46ab-9edb-78ea467367eb-kube-api-access-d56nv\") pod \"maas-api-revoke-keys-e2e-worker-w1-d797d2-cplms\" (UID: \"c960481b-979f-46ab-9edb-78ea467367eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-d797d2-cplms"
Oct 30 19:47:31.503414 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:31.503393    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c960481b-979f-46ab-9edb-78ea467367eb-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-d797d2-cplms\" (UID: \"c960481b-979f-46ab-9edb-78ea467367eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-d797d2-cplms"
Oct 30 19:47:31.543023 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:31.542990    5907 patch_prober.go:29] interesting pod/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:47:31.543199 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:31.543042    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd" podUID="dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:47:31.604491 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:31.604445    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c960481b-979f-46ab-9edb-78ea467367eb-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-d797d2-cplms\" (UID: \"c960481b-979f-46ab-9edb-78ea467367eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-d797d2-cplms"
Oct 30 19:47:31.604676 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:31.604525    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c960481b-979f-46ab-9edb-78ea467367eb-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-d797d2-cplms\" (UID: \"c960481b-979f-46ab-9edb-78ea467367eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-d797d2-cplms"
Oct 30 19:47:31.604676 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:31.604667    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d56nv\" (UniqueName: \"kubernetes.io/projected/c960481b-979f-46ab-9edb-78ea467367eb-kube-api-access-d56nv\") pod \"maas-api-revoke-keys-e2e-worker-w1-d797d2-cplms\" (UID: \"c960481b-979f-46ab-9edb-78ea467367eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-d797d2-cplms"
Oct 30 19:47:31.605203 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:31.605176    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c960481b-979f-46ab-9edb-78ea467367eb-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-d797d2-cplms\" (UID: \"c960481b-979f-46ab-9edb-78ea467367eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-d797d2-cplms"
Oct 30 19:47:31.607055 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:31.607034    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c960481b-979f-46ab-9edb-78ea467367eb-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-d797d2-cplms\" (UID: \"c960481b-979f-46ab-9edb-78ea467367eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-d797d2-cplms"
Oct 30 19:47:31.626006 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:31.625977    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d56nv\" (UniqueName: \"kubernetes.io/projected/c960481b-979f-46ab-9edb-78ea467367eb-kube-api-access-d56nv\") pod \"maas-api-revoke-keys-e2e-worker-w1-d797d2-cplms\" (UID: \"c960481b-979f-46ab-9edb-78ea467367eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-d797d2-cplms"
Oct 30 19:47:31.779169 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:31.779089    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-d797d2-cplms"
Oct 30 19:47:32.141140 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:32.141106    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-d797d2-cplms"]
Oct 30 19:47:32.141672 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:47:32.141642    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc960481b_979f_46ab_9edb_78ea467367eb.slice/crio-d1db1db0724b3b591346bcb04d7584a6b03ed03564ed8e4f88698a622f01e0dd WatchSource:0}: Error finding container d1db1db0724b3b591346bcb04d7584a6b03ed03564ed8e4f88698a622f01e0dd: Status 404 returned error can't find the container with id d1db1db0724b3b591346bcb04d7584a6b03ed03564ed8e4f88698a622f01e0dd
Oct 30 19:47:32.650550 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:32.650514    5907 generic.go:356] "Generic (PLEG): container finished" podID="0435effb-f3d6-450f-beed-937cec7badb0" containerID="4be7617af316401736a1464fcac5393879ac9028143df75d2beae64a22f2660d" exitCode=0
Oct 30 19:47:32.650700 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:32.650586    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8" event={"ID":"0435effb-f3d6-450f-beed-937cec7badb0","Type":"ContainerDied","Data":"4be7617af316401736a1464fcac5393879ac9028143df75d2beae64a22f2660d"}
Oct 30 19:47:32.650700 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:32.650662    5907 scope.go:122] "RemoveContainer" containerID="e26d6b861043f3fc8b434d4774de412d9c28fc362a35d3c2135c4569742f2e00"
Oct 30 19:47:32.652557 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:32.652532    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-d797d2-cplms" event={"ID":"c960481b-979f-46ab-9edb-78ea467367eb","Type":"ContainerStarted","Data":"9a570bbcc2174846b7fd60475f46e6376306c410a0f0cd9c99ea2bb109516c0d"}
Oct 30 19:47:32.652681 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:32.652584    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-d797d2-cplms" event={"ID":"c960481b-979f-46ab-9edb-78ea467367eb","Type":"ContainerStarted","Data":"d1db1db0724b3b591346bcb04d7584a6b03ed03564ed8e4f88698a622f01e0dd"}
Oct 30 19:47:32.678769 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:32.678748    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:47:32.697588 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:32.697518    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-d797d2-cplms" podStartSLOduration=1.697495559 podStartE2EDuration="1.697495559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:47:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:32.697352185 +0000 UTC m=+1564.540284125" watchObservedRunningTime="2026-10-30 19:47:32.697495559 +0000 UTC m=+1564.540427500"
Oct 30 19:47:32.738307 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:32.738274    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0435effb-f3d6-450f-beed-937cec7badb0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0435effb-f3d6-450f-beed-937cec7badb0-workload-certs\") pod \"0435effb-f3d6-450f-beed-937cec7badb0\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") "
Oct 30 19:47:32.738521 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:32.738328    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0435effb-f3d6-450f-beed-937cec7badb0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0435effb-f3d6-450f-beed-937cec7badb0-istio-envoy\") pod \"0435effb-f3d6-450f-beed-937cec7badb0\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") "
Oct 30 19:47:32.738521 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:32.738388    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0435effb-f3d6-450f-beed-937cec7badb0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0435effb-f3d6-450f-beed-937cec7badb0-istio-data\") pod \"0435effb-f3d6-450f-beed-937cec7badb0\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") "
Oct 30 19:47:32.738521 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:32.738463    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0435effb-f3d6-450f-beed-937cec7badb0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0435effb-f3d6-450f-beed-937cec7badb0-istio-podinfo\") pod \"0435effb-f3d6-450f-beed-937cec7badb0\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") "
Oct 30 19:47:32.738521 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:32.738515    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0435effb-f3d6-450f-beed-937cec7badb0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0435effb-f3d6-450f-beed-937cec7badb0-workload-socket\") pod \"0435effb-f3d6-450f-beed-937cec7badb0\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") "
Oct 30 19:47:32.738766 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:32.738571    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0435effb-f3d6-450f-beed-937cec7badb0-istio-token\" (UniqueName: \"kubernetes.io/projected/0435effb-f3d6-450f-beed-937cec7badb0-istio-token\") pod \"0435effb-f3d6-450f-beed-937cec7badb0\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") "
Oct 30 19:47:32.738766 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:32.738626    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0435effb-f3d6-450f-beed-937cec7badb0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0435effb-f3d6-450f-beed-937cec7badb0-credential-socket\") pod \"0435effb-f3d6-450f-beed-937cec7badb0\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") "
Oct 30 19:47:32.738766 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:32.738626    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0435effb-f3d6-450f-beed-937cec7badb0-workload-certs" pod "0435effb-f3d6-450f-beed-937cec7badb0" (UID: "0435effb-f3d6-450f-beed-937cec7badb0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:32.738766 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:32.738707    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0435effb-f3d6-450f-beed-937cec7badb0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0435effb-f3d6-450f-beed-937cec7badb0-istiod-ca-cert\") pod \"0435effb-f3d6-450f-beed-937cec7badb0\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") "
Oct 30 19:47:32.739057 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:32.738766    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0435effb-f3d6-450f-beed-937cec7badb0-kube-api-access-glklx\" (UniqueName: \"kubernetes.io/projected/0435effb-f3d6-450f-beed-937cec7badb0-kube-api-access-glklx\") pod \"0435effb-f3d6-450f-beed-937cec7badb0\" (UID: \"0435effb-f3d6-450f-beed-937cec7badb0\") "
Oct 30 19:47:32.739057 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:32.739035    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0435effb-f3d6-450f-beed-937cec7badb0-istio-data" pod "0435effb-f3d6-450f-beed-937cec7badb0" (UID: "0435effb-f3d6-450f-beed-937cec7badb0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:32.739547 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:32.739519    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0435effb-f3d6-450f-beed-937cec7badb0-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:32.739653 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:32.739551    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0435effb-f3d6-450f-beed-937cec7badb0-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:32.740051 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:32.740023    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0435effb-f3d6-450f-beed-937cec7badb0-credential-socket" pod "0435effb-f3d6-450f-beed-937cec7badb0" (UID: "0435effb-f3d6-450f-beed-937cec7badb0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:32.740351 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:32.740326    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0435effb-f3d6-450f-beed-937cec7badb0-istiod-ca-cert" pod "0435effb-f3d6-450f-beed-937cec7badb0" (UID: "0435effb-f3d6-450f-beed-937cec7badb0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:47:32.740534 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:32.740511    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0435effb-f3d6-450f-beed-937cec7badb0-workload-socket" pod "0435effb-f3d6-450f-beed-937cec7badb0" (UID: "0435effb-f3d6-450f-beed-937cec7badb0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:32.741605 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:32.741578    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0435effb-f3d6-450f-beed-937cec7badb0-istio-envoy" pod "0435effb-f3d6-450f-beed-937cec7badb0" (UID: "0435effb-f3d6-450f-beed-937cec7badb0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:32.742592 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:32.742572    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0435effb-f3d6-450f-beed-937cec7badb0-istio-podinfo" pod "0435effb-f3d6-450f-beed-937cec7badb0" (UID: "0435effb-f3d6-450f-beed-937cec7badb0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:47:32.744028 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:32.744007    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0435effb-f3d6-450f-beed-937cec7badb0-kube-api-access-glklx" pod "0435effb-f3d6-450f-beed-937cec7badb0" (UID: "0435effb-f3d6-450f-beed-937cec7badb0"). InnerVolumeSpecName "kube-api-access-glklx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:32.750319 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:32.750294    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0435effb-f3d6-450f-beed-937cec7badb0-istio-token" pod "0435effb-f3d6-450f-beed-937cec7badb0" (UID: "0435effb-f3d6-450f-beed-937cec7badb0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:32.841246 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:32.841213    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0435effb-f3d6-450f-beed-937cec7badb0-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:32.841246 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:32.841241    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0435effb-f3d6-450f-beed-937cec7badb0-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:32.841246 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:32.841250    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0435effb-f3d6-450f-beed-937cec7badb0-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:32.841767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:32.841259    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glklx\" (UniqueName: \"kubernetes.io/projected/0435effb-f3d6-450f-beed-937cec7badb0-kube-api-access-glklx\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:32.841767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:32.841268    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0435effb-f3d6-450f-beed-937cec7badb0-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:32.841767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:32.841276    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0435effb-f3d6-450f-beed-937cec7badb0-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:32.841767 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:32.841283    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0435effb-f3d6-450f-beed-937cec7badb0-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:33.167897 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.167837    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d04d31e9-592l9"]
Oct 30 19:47:33.168474 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.168459    5907 cpu_manager.go:414] "containerMap: removing container" podUID="0435effb-f3d6-450f-beed-937cec7badb0" containerName="istio-proxy"
Oct 30 19:47:33.168519 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.168477    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0435effb-f3d6-450f-beed-937cec7badb0" containerName="istio-proxy"
Oct 30 19:47:33.168519 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.168500    5907 cpu_manager.go:414] "containerMap: removing container" podUID="0435effb-f3d6-450f-beed-937cec7badb0" containerName="istio-proxy"
Oct 30 19:47:33.168519 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.168505    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0435effb-f3d6-450f-beed-937cec7badb0" containerName="istio-proxy"
Oct 30 19:47:33.168623 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.168543    5907 cpu_manager.go:414] "containerMap: removing container" podUID="0da36910-7634-4556-b95b-451340c2a584" containerName="istio-proxy"
Oct 30 19:47:33.168623 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.168550    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0da36910-7634-4556-b95b-451340c2a584" containerName="istio-proxy"
Oct 30 19:47:33.168693 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.168655    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="0da36910-7634-4556-b95b-451340c2a584" containerName="istio-proxy"
Oct 30 19:47:33.168693 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.168666    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="0435effb-f3d6-450f-beed-937cec7badb0" containerName="istio-proxy"
Oct 30 19:47:33.168693 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.168676    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="0435effb-f3d6-450f-beed-937cec7badb0" containerName="istio-proxy"
Oct 30 19:47:33.172013 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.171991    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d04d31e9-592l9"
Oct 30 19:47:33.191069 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.191042    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-112c28ee-bmzsd"]
Oct 30 19:47:33.195330 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.195305    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d04d31e9-592l9"]
Oct 30 19:47:33.195495 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.195427    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-112c28ee-bmzsd"
Oct 30 19:47:33.223905 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.221963    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-112c28ee-bmzsd"]
Oct 30 19:47:33.246969 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.246933    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7760da1a-a95e-4f3e-9512-08cb4e523cd7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d04d31e9-592l9\" (UID: \"7760da1a-a95e-4f3e-9512-08cb4e523cd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d04d31e9-592l9"
Oct 30 19:47:33.247109 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.247071    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqtxv\" (UniqueName: \"kubernetes.io/projected/7760da1a-a95e-4f3e-9512-08cb4e523cd7-kube-api-access-pqtxv\") pod \"maas-api-revoke-sub-d04d31e9-592l9\" (UID: \"7760da1a-a95e-4f3e-9512-08cb4e523cd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d04d31e9-592l9"
Oct 30 19:47:33.247148 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.247104    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b2fa1172-31da-4cab-9622-f4adc52c0ca2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-112c28ee-bmzsd\" (UID: \"b2fa1172-31da-4cab-9622-f4adc52c0ca2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-112c28ee-bmzsd"
Oct 30 19:47:33.247148 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.247131    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b2fa1172-31da-4cab-9622-f4adc52c0ca2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-112c28ee-bmzsd\" (UID: \"b2fa1172-31da-4cab-9622-f4adc52c0ca2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-112c28ee-bmzsd"
Oct 30 19:47:33.247356 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.247330    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7760da1a-a95e-4f3e-9512-08cb4e523cd7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d04d31e9-592l9\" (UID: \"7760da1a-a95e-4f3e-9512-08cb4e523cd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d04d31e9-592l9"
Oct 30 19:47:33.247401 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.247390    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sxt5\" (UniqueName: \"kubernetes.io/projected/b2fa1172-31da-4cab-9622-f4adc52c0ca2-kube-api-access-4sxt5\") pod \"maas-api-revoke-sub-112c28ee-bmzsd\" (UID: \"b2fa1172-31da-4cab-9622-f4adc52c0ca2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-112c28ee-bmzsd"
Oct 30 19:47:33.348985 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.348895    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7760da1a-a95e-4f3e-9512-08cb4e523cd7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d04d31e9-592l9\" (UID: \"7760da1a-a95e-4f3e-9512-08cb4e523cd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d04d31e9-592l9"
Oct 30 19:47:33.349161 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.349018    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sxt5\" (UniqueName: \"kubernetes.io/projected/b2fa1172-31da-4cab-9622-f4adc52c0ca2-kube-api-access-4sxt5\") pod \"maas-api-revoke-sub-112c28ee-bmzsd\" (UID: \"b2fa1172-31da-4cab-9622-f4adc52c0ca2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-112c28ee-bmzsd"
Oct 30 19:47:33.349161 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.349069    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7760da1a-a95e-4f3e-9512-08cb4e523cd7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d04d31e9-592l9\" (UID: \"7760da1a-a95e-4f3e-9512-08cb4e523cd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d04d31e9-592l9"
Oct 30 19:47:33.349298 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.349211    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqtxv\" (UniqueName: \"kubernetes.io/projected/7760da1a-a95e-4f3e-9512-08cb4e523cd7-kube-api-access-pqtxv\") pod \"maas-api-revoke-sub-d04d31e9-592l9\" (UID: \"7760da1a-a95e-4f3e-9512-08cb4e523cd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d04d31e9-592l9"
Oct 30 19:47:33.349298 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.349255    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b2fa1172-31da-4cab-9622-f4adc52c0ca2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-112c28ee-bmzsd\" (UID: \"b2fa1172-31da-4cab-9622-f4adc52c0ca2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-112c28ee-bmzsd"
Oct 30 19:47:33.349298 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.349286    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b2fa1172-31da-4cab-9622-f4adc52c0ca2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-112c28ee-bmzsd\" (UID: \"b2fa1172-31da-4cab-9622-f4adc52c0ca2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-112c28ee-bmzsd"
Oct 30 19:47:33.349922 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.349853    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7760da1a-a95e-4f3e-9512-08cb4e523cd7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d04d31e9-592l9\" (UID: \"7760da1a-a95e-4f3e-9512-08cb4e523cd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d04d31e9-592l9"
Oct 30 19:47:33.350185 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.350131    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b2fa1172-31da-4cab-9622-f4adc52c0ca2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-112c28ee-bmzsd\" (UID: \"b2fa1172-31da-4cab-9622-f4adc52c0ca2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-112c28ee-bmzsd"
Oct 30 19:47:33.352041 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.352018    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7760da1a-a95e-4f3e-9512-08cb4e523cd7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d04d31e9-592l9\" (UID: \"7760da1a-a95e-4f3e-9512-08cb4e523cd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d04d31e9-592l9"
Oct 30 19:47:33.352186 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.352159    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b2fa1172-31da-4cab-9622-f4adc52c0ca2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-112c28ee-bmzsd\" (UID: \"b2fa1172-31da-4cab-9622-f4adc52c0ca2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-112c28ee-bmzsd"
Oct 30 19:47:33.371167 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.371109    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sxt5\" (UniqueName: \"kubernetes.io/projected/b2fa1172-31da-4cab-9622-f4adc52c0ca2-kube-api-access-4sxt5\") pod \"maas-api-revoke-sub-112c28ee-bmzsd\" (UID: \"b2fa1172-31da-4cab-9622-f4adc52c0ca2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-112c28ee-bmzsd"
Oct 30 19:47:33.371350 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.371315    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqtxv\" (UniqueName: \"kubernetes.io/projected/7760da1a-a95e-4f3e-9512-08cb4e523cd7-kube-api-access-pqtxv\") pod \"maas-api-revoke-sub-d04d31e9-592l9\" (UID: \"7760da1a-a95e-4f3e-9512-08cb4e523cd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d04d31e9-592l9"
Oct 30 19:47:33.486422 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.486390    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d04d31e9-592l9"
Oct 30 19:47:33.511361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.511333    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-112c28ee-bmzsd"
Oct 30 19:47:33.703206 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.703171    5907 generic.go:356] "Generic (PLEG): container finished" podID="c960481b-979f-46ab-9edb-78ea467367eb" containerID="9a570bbcc2174846b7fd60475f46e6376306c410a0f0cd9c99ea2bb109516c0d" exitCode=0
Oct 30 19:47:33.703404 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.703252    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-d797d2-cplms" event={"ID":"c960481b-979f-46ab-9edb-78ea467367eb","Type":"ContainerDied","Data":"9a570bbcc2174846b7fd60475f46e6376306c410a0f0cd9c99ea2bb109516c0d"}
Oct 30 19:47:33.707890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.704995    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8" event={"ID":"0435effb-f3d6-450f-beed-937cec7badb0","Type":"ContainerDied","Data":"352dcd587e8c2693387ea6c97d091395ae229e2a1f2e919e8c9d641aadb07cd0"}
Oct 30 19:47:33.707890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.705042    5907 scope.go:122] "RemoveContainer" containerID="4be7617af316401736a1464fcac5393879ac9028143df75d2beae64a22f2660d"
Oct 30 19:47:33.707890 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.705005    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"
Oct 30 19:47:33.808032 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.808003    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d04d31e9-592l9"]
Oct 30 19:47:33.819025 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:47:33.818986    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7760da1a_a95e_4f3e_9512_08cb4e523cd7.slice/crio-f287aaabc2a22b33261c7f38d46f7ff1f8ecfc8e20b6bc917aa7a678a89d6d7e WatchSource:0}: Error finding container f287aaabc2a22b33261c7f38d46f7ff1f8ecfc8e20b6bc917aa7a678a89d6d7e: Status 404 returned error can't find the container with id f287aaabc2a22b33261c7f38d46f7ff1f8ecfc8e20b6bc917aa7a678a89d6d7e
Oct 30 19:47:33.829396 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.829372    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"]
Oct 30 19:47:33.838545 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.838508    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-67953efd-openshift-default-f666c5445-7cxn8"]
Oct 30 19:47:33.847573 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.847551    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-112c28ee-bmzsd"]
Oct 30 19:47:33.849526 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:47:33.849494    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2fa1172_31da_4cab_9622_f4adc52c0ca2.slice/crio-d7e7b1511617aa5326da0431359bb4ff0f070eb344c29f49701b63840b6d401e WatchSource:0}: Error finding container d7e7b1511617aa5326da0431359bb4ff0f070eb344c29f49701b63840b6d401e: Status 404 returned error can't find the container with id d7e7b1511617aa5326da0431359bb4ff0f070eb344c29f49701b63840b6d401e
Oct 30 19:47:33.861193 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:33.861165    5907 scope.go:122] "RemoveContainer" containerID="4f0094a43cda37c653f36c66af261a08a4f80ef27bb102b24b984e9a4ab04e96"
Oct 30 19:47:34.266171 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:34.266136    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main" containerID="cri-o://6a2fdccab00b205692e7d3de2df0b12e6103a08ed0d329b9e73f151942a39431" gracePeriod=45
Oct 30 19:47:34.289707 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:34.289680    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0435effb-f3d6-450f-beed-937cec7badb0" path="/var/lib/kubelet/pods/0435effb-f3d6-450f-beed-937cec7badb0/volumes"
Oct 30 19:47:34.578823 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:34.578496    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:47:34.679811 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:34.678152    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d1837f0-48b2-446c-946c-4007adf95814-kube-api-access-msfkh\" (UniqueName: \"kubernetes.io/projected/7d1837f0-48b2-446c-946c-4007adf95814-kube-api-access-msfkh\") pod \"7d1837f0-48b2-446c-946c-4007adf95814\" (UID: \"7d1837f0-48b2-446c-946c-4007adf95814\") "
Oct 30 19:47:34.679811 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:34.678315    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d1837f0-48b2-446c-946c-4007adf95814-home\" (UniqueName: \"kubernetes.io/empty-dir/7d1837f0-48b2-446c-946c-4007adf95814-home\") pod \"7d1837f0-48b2-446c-946c-4007adf95814\" (UID: \"7d1837f0-48b2-446c-946c-4007adf95814\") "
Oct 30 19:47:34.679811 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:34.678376    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d1837f0-48b2-446c-946c-4007adf95814-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d1837f0-48b2-446c-946c-4007adf95814-tmp-dir\") pod \"7d1837f0-48b2-446c-946c-4007adf95814\" (UID: \"7d1837f0-48b2-446c-946c-4007adf95814\") "
Oct 30 19:47:34.679811 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:34.678411    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7d1837f0-48b2-446c-946c-4007adf95814-tls-certs\" (UniqueName: \"kubernetes.io/secret/7d1837f0-48b2-446c-946c-4007adf95814-tls-certs\") pod \"7d1837f0-48b2-446c-946c-4007adf95814\" (UID: \"7d1837f0-48b2-446c-946c-4007adf95814\") "
Oct 30 19:47:34.679811 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:34.678473    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d1837f0-48b2-446c-946c-4007adf95814-dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d1837f0-48b2-446c-946c-4007adf95814-dshm\") pod \"7d1837f0-48b2-446c-946c-4007adf95814\" (UID: \"7d1837f0-48b2-446c-946c-4007adf95814\") "
Oct 30 19:47:34.679811 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:34.678509    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d1837f0-48b2-446c-946c-4007adf95814-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d1837f0-48b2-446c-946c-4007adf95814-model-cache\") pod \"7d1837f0-48b2-446c-946c-4007adf95814\" (UID: \"7d1837f0-48b2-446c-946c-4007adf95814\") "
Oct 30 19:47:34.679811 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:34.679412    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d1837f0-48b2-446c-946c-4007adf95814-model-cache" pod "7d1837f0-48b2-446c-946c-4007adf95814" (UID: "7d1837f0-48b2-446c-946c-4007adf95814"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:34.680375 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:34.680030    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d1837f0-48b2-446c-946c-4007adf95814-tmp-dir" pod "7d1837f0-48b2-446c-946c-4007adf95814" (UID: "7d1837f0-48b2-446c-946c-4007adf95814"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:34.680375 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:34.680269    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d1837f0-48b2-446c-946c-4007adf95814-home" pod "7d1837f0-48b2-446c-946c-4007adf95814" (UID: "7d1837f0-48b2-446c-946c-4007adf95814"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:34.727403 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:34.726021    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-112c28ee-bmzsd" event={"ID":"b2fa1172-31da-4cab-9622-f4adc52c0ca2","Type":"ContainerStarted","Data":"15c522243d334731ab97cfb4b7ed018144321739bc94ffaa8992551c3b42699b"}
Oct 30 19:47:34.727403 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:34.726091    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-112c28ee-bmzsd" event={"ID":"b2fa1172-31da-4cab-9622-f4adc52c0ca2","Type":"ContainerStarted","Data":"d7e7b1511617aa5326da0431359bb4ff0f070eb344c29f49701b63840b6d401e"}
Oct 30 19:47:34.730277 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:34.730245    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d04d31e9-592l9" event={"ID":"7760da1a-a95e-4f3e-9512-08cb4e523cd7","Type":"ContainerStarted","Data":"34dafadcc865c56cb5dcc851a407f88348f989c632ef48d5c528518b30827cb7"}
Oct 30 19:47:34.730415 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:34.730303    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d04d31e9-592l9" event={"ID":"7760da1a-a95e-4f3e-9512-08cb4e523cd7","Type":"ContainerStarted","Data":"f287aaabc2a22b33261c7f38d46f7ff1f8ecfc8e20b6bc917aa7a678a89d6d7e"}
Oct 30 19:47:34.734752 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:34.734378    5907 generic.go:356] "Generic (PLEG): container finished" podID="7d1837f0-48b2-446c-946c-4007adf95814" containerID="6a2fdccab00b205692e7d3de2df0b12e6103a08ed0d329b9e73f151942a39431" exitCode=0
Oct 30 19:47:34.734929 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:34.734769    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"
Oct 30 19:47:34.735012 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:34.734922    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" event={"ID":"7d1837f0-48b2-446c-946c-4007adf95814","Type":"ContainerDied","Data":"6a2fdccab00b205692e7d3de2df0b12e6103a08ed0d329b9e73f151942a39431"}
Oct 30 19:47:34.735012 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:34.734955    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b" event={"ID":"7d1837f0-48b2-446c-946c-4007adf95814","Type":"ContainerDied","Data":"80702fbc2cb7894232eadbb8ccda5d94d9b128a9874da474e247c0a1149566aa"}
Oct 30 19:47:34.735012 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:34.734988    5907 scope.go:122] "RemoveContainer" containerID="6a2fdccab00b205692e7d3de2df0b12e6103a08ed0d329b9e73f151942a39431"
Oct 30 19:47:34.743496 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:34.742592    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d1837f0-48b2-446c-946c-4007adf95814-tls-certs" pod "7d1837f0-48b2-446c-946c-4007adf95814" (UID: "7d1837f0-48b2-446c-946c-4007adf95814"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:47:34.743496 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:34.742744    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d1837f0-48b2-446c-946c-4007adf95814-kube-api-access-msfkh" pod "7d1837f0-48b2-446c-946c-4007adf95814" (UID: "7d1837f0-48b2-446c-946c-4007adf95814"). InnerVolumeSpecName "kube-api-access-msfkh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:34.754466 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:34.754419    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d1837f0-48b2-446c-946c-4007adf95814-dshm" pod "7d1837f0-48b2-446c-946c-4007adf95814" (UID: "7d1837f0-48b2-446c-946c-4007adf95814"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:34.773046 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:34.771583    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-112c28ee-bmzsd" podStartSLOduration=1.771562316 podStartE2EDuration="1.771562316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:47:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:34.770142576 +0000 UTC m=+1566.613074516" watchObservedRunningTime="2026-10-30 19:47:34.771562316 +0000 UTC m=+1566.614494256"
Oct 30 19:47:34.776826 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:34.776164    5907 scope.go:122] "RemoveContainer" containerID="6a2fdccab00b205692e7d3de2df0b12e6103a08ed0d329b9e73f151942a39431"
Oct 30 19:47:34.776826 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:47:34.776701    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a2fdccab00b205692e7d3de2df0b12e6103a08ed0d329b9e73f151942a39431\": container with ID starting with 6a2fdccab00b205692e7d3de2df0b12e6103a08ed0d329b9e73f151942a39431 not found: ID does not exist" containerID="6a2fdccab00b205692e7d3de2df0b12e6103a08ed0d329b9e73f151942a39431"
Oct 30 19:47:34.776826 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:34.776739    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a2fdccab00b205692e7d3de2df0b12e6103a08ed0d329b9e73f151942a39431"} err="failed to get container status \"6a2fdccab00b205692e7d3de2df0b12e6103a08ed0d329b9e73f151942a39431\": rpc error: code = NotFound desc = could not find container \"6a2fdccab00b205692e7d3de2df0b12e6103a08ed0d329b9e73f151942a39431\": container with ID starting with 6a2fdccab00b205692e7d3de2df0b12e6103a08ed0d329b9e73f151942a39431 not found: ID does not exist"
Oct 30 19:47:34.781236 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:34.781071    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msfkh\" (UniqueName: \"kubernetes.io/projected/7d1837f0-48b2-446c-946c-4007adf95814-kube-api-access-msfkh\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:34.781236 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:34.781103    5907 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d1837f0-48b2-446c-946c-4007adf95814-home\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:34.781236 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:34.781145    5907 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d1837f0-48b2-446c-946c-4007adf95814-tmp-dir\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:34.781236 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:34.781161    5907 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d1837f0-48b2-446c-946c-4007adf95814-tls-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:34.781236 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:34.781173    5907 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d1837f0-48b2-446c-946c-4007adf95814-dshm\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:34.781236 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:34.781186    5907 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d1837f0-48b2-446c-946c-4007adf95814-model-cache\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:34.851377 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:34.849887    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d04d31e9-592l9" podStartSLOduration=1.849837419 podStartE2EDuration="1.849837419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:47:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:34.837341651 +0000 UTC m=+1566.680273594" watchObservedRunningTime="2026-10-30 19:47:34.849837419 +0000 UTC m=+1566.692769359"
Oct 30 19:47:35.024395 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:35.024365    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-d797d2-cplms"
Oct 30 19:47:35.113191 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:35.113150    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"]
Oct 30 19:47:35.118731 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:35.118682    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-f913c6/distinct-2-w4-f913c6-kserve-6cf78f8886-vwn8b"]
Oct 30 19:47:35.189043 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:35.188984    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c960481b-979f-46ab-9edb-78ea467367eb-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c960481b-979f-46ab-9edb-78ea467367eb-maas-api-xks-ca\") pod \"c960481b-979f-46ab-9edb-78ea467367eb\" (UID: \"c960481b-979f-46ab-9edb-78ea467367eb\") "
Oct 30 19:47:35.189268 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:35.189070    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c960481b-979f-46ab-9edb-78ea467367eb-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c960481b-979f-46ab-9edb-78ea467367eb-maas-api-openshift-ca\") pod \"c960481b-979f-46ab-9edb-78ea467367eb\" (UID: \"c960481b-979f-46ab-9edb-78ea467367eb\") "
Oct 30 19:47:35.189268 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:35.189235    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c960481b-979f-46ab-9edb-78ea467367eb-kube-api-access-d56nv\" (UniqueName: \"kubernetes.io/projected/c960481b-979f-46ab-9edb-78ea467367eb-kube-api-access-d56nv\") pod \"c960481b-979f-46ab-9edb-78ea467367eb\" (UID: \"c960481b-979f-46ab-9edb-78ea467367eb\") "
Oct 30 19:47:35.190073 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:35.190043    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c960481b-979f-46ab-9edb-78ea467367eb-maas-api-openshift-ca" pod "c960481b-979f-46ab-9edb-78ea467367eb" (UID: "c960481b-979f-46ab-9edb-78ea467367eb"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:47:35.191954 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:35.191914    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c960481b-979f-46ab-9edb-78ea467367eb-maas-api-xks-ca" pod "c960481b-979f-46ab-9edb-78ea467367eb" (UID: "c960481b-979f-46ab-9edb-78ea467367eb"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:47:35.192083 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:35.192017    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c960481b-979f-46ab-9edb-78ea467367eb-kube-api-access-d56nv" pod "c960481b-979f-46ab-9edb-78ea467367eb" (UID: "c960481b-979f-46ab-9edb-78ea467367eb"). InnerVolumeSpecName "kube-api-access-d56nv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:35.290930 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:35.290835    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c960481b-979f-46ab-9edb-78ea467367eb-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:35.290930 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:35.290927    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d56nv\" (UniqueName: \"kubernetes.io/projected/c960481b-979f-46ab-9edb-78ea467367eb-kube-api-access-d56nv\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:35.291221 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:35.290950    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c960481b-979f-46ab-9edb-78ea467367eb-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:35.812059 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:35.812005    5907 generic.go:356] "Generic (PLEG): container finished" podID="b2fa1172-31da-4cab-9622-f4adc52c0ca2" containerID="15c522243d334731ab97cfb4b7ed018144321739bc94ffaa8992551c3b42699b" exitCode=0
Oct 30 19:47:35.815642 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:35.815546    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-112c28ee-bmzsd" event={"ID":"b2fa1172-31da-4cab-9622-f4adc52c0ca2","Type":"ContainerDied","Data":"15c522243d334731ab97cfb4b7ed018144321739bc94ffaa8992551c3b42699b"}
Oct 30 19:47:35.818380 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:35.816505    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-d797d2-cplms" event={"ID":"c960481b-979f-46ab-9edb-78ea467367eb","Type":"ContainerDied","Data":"d1db1db0724b3b591346bcb04d7584a6b03ed03564ed8e4f88698a622f01e0dd"}
Oct 30 19:47:35.818380 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:35.816551    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d1db1db0724b3b591346bcb04d7584a6b03ed03564ed8e4f88698a622f01e0dd"
Oct 30 19:47:35.818380 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:35.816619    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-d797d2-cplms"
Oct 30 19:47:35.853783 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:35.851735    5907 generic.go:356] "Generic (PLEG): container finished" podID="7760da1a-a95e-4f3e-9512-08cb4e523cd7" containerID="34dafadcc865c56cb5dcc851a407f88348f989c632ef48d5c528518b30827cb7" exitCode=0
Oct 30 19:47:35.853783 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:35.851830    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d04d31e9-592l9" event={"ID":"7760da1a-a95e-4f3e-9512-08cb4e523cd7","Type":"ContainerDied","Data":"34dafadcc865c56cb5dcc851a407f88348f989c632ef48d5c528518b30827cb7"}
Oct 30 19:47:36.377891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:36.374188    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7d1837f0-48b2-446c-946c-4007adf95814" path="/var/lib/kubelet/pods/7d1837f0-48b2-446c-946c-4007adf95814/volumes"
Oct 30 19:47:37.083886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.080262    5907 generic.go:356] "Generic (PLEG): container finished" podID="dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe" containerID="77b47da55d91f085d97dca9028eac1585e92bd7f70c3e1fe58186110e915098d" exitCode=0
Oct 30 19:47:37.083886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.081050    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd" event={"ID":"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe","Type":"ContainerDied","Data":"77b47da55d91f085d97dca9028eac1585e92bd7f70c3e1fe58186110e915098d"}
Oct 30 19:47:37.083886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.081214    5907 scope.go:122] "RemoveContainer" containerID="45500cbe0d670569b588d9d75bd90ea5d62448e8a0c448d49058575d078886f5"
Oct 30 19:47:37.096971 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.092421    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:47:37.160758 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.160663    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-credential-socket\") pod \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") "
Oct 30 19:47:37.161048 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.161022    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-istio-token\" (UniqueName: \"kubernetes.io/projected/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-istio-token\") pod \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") "
Oct 30 19:47:37.165990 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.165961    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-istio-podinfo\") pod \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") "
Oct 30 19:47:37.166224 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.166205    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-istio-data\") pod \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") "
Oct 30 19:47:37.166409 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.166391    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-kube-api-access-dndc4\" (UniqueName: \"kubernetes.io/projected/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-kube-api-access-dndc4\") pod \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") "
Oct 30 19:47:37.166551 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.166535    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-workload-socket\") pod \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") "
Oct 30 19:47:37.166725 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.166706    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-workload-certs\") pod \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") "
Oct 30 19:47:37.166851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.166836    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-istio-envoy\") pod \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") "
Oct 30 19:47:37.180984 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.180957    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-istiod-ca-cert\") pod \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\" (UID: \"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe\") "
Oct 30 19:47:37.182478 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.182442    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-istio-data" pod "dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe" (UID: "dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:37.182808 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.182788    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-credential-socket" pod "dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe" (UID: "dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:37.183416 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.183393    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-istiod-ca-cert" pod "dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe" (UID: "dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:47:37.187400 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.187358    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-workload-socket" pod "dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe" (UID: "dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:37.213374 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.210956    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-workload-certs" pod "dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe" (UID: "dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:37.325904 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.298632    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:37.326200 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.326172    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:37.326318 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.326304    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:37.326453 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.326438    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:37.326557 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.326545    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:37.397166 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.394036    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-istio-token" pod "dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe" (UID: "dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:37.398209 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.398173    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-istio-envoy" pod "dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe" (UID: "dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:37.401500 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.401456    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-kube-api-access-dndc4" pod "dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe" (UID: "dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe"). InnerVolumeSpecName "kube-api-access-dndc4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:37.402355 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.402320    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-istio-podinfo" pod "dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe" (UID: "dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:47:37.434887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.433415    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:37.434887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.433447    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:37.434887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.433462    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dndc4\" (UniqueName: \"kubernetes.io/projected/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-kube-api-access-dndc4\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:37.434887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.433477    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:37.534113 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.534078    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d04d31e9-592l9"
Oct 30 19:47:37.569536 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.569494    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-112c28ee-bmzsd"
Oct 30 19:47:37.635782 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.635748    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b2fa1172-31da-4cab-9622-f4adc52c0ca2-kube-api-access-4sxt5\" (UniqueName: \"kubernetes.io/projected/b2fa1172-31da-4cab-9622-f4adc52c0ca2-kube-api-access-4sxt5\") pod \"b2fa1172-31da-4cab-9622-f4adc52c0ca2\" (UID: \"b2fa1172-31da-4cab-9622-f4adc52c0ca2\") "
Oct 30 19:47:37.635979 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.635796    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b2fa1172-31da-4cab-9622-f4adc52c0ca2-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b2fa1172-31da-4cab-9622-f4adc52c0ca2-maas-api-xks-ca\") pod \"b2fa1172-31da-4cab-9622-f4adc52c0ca2\" (UID: \"b2fa1172-31da-4cab-9622-f4adc52c0ca2\") "
Oct 30 19:47:37.635979 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.635844    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b2fa1172-31da-4cab-9622-f4adc52c0ca2-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b2fa1172-31da-4cab-9622-f4adc52c0ca2-maas-api-openshift-ca\") pod \"b2fa1172-31da-4cab-9622-f4adc52c0ca2\" (UID: \"b2fa1172-31da-4cab-9622-f4adc52c0ca2\") "
Oct 30 19:47:37.636120 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.636016    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7760da1a-a95e-4f3e-9512-08cb4e523cd7-kube-api-access-pqtxv\" (UniqueName: \"kubernetes.io/projected/7760da1a-a95e-4f3e-9512-08cb4e523cd7-kube-api-access-pqtxv\") pod \"7760da1a-a95e-4f3e-9512-08cb4e523cd7\" (UID: \"7760da1a-a95e-4f3e-9512-08cb4e523cd7\") "
Oct 30 19:47:37.636120 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.636088    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7760da1a-a95e-4f3e-9512-08cb4e523cd7-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7760da1a-a95e-4f3e-9512-08cb4e523cd7-maas-api-xks-ca\") pod \"7760da1a-a95e-4f3e-9512-08cb4e523cd7\" (UID: \"7760da1a-a95e-4f3e-9512-08cb4e523cd7\") "
Oct 30 19:47:37.636228 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.636121    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7760da1a-a95e-4f3e-9512-08cb4e523cd7-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7760da1a-a95e-4f3e-9512-08cb4e523cd7-maas-api-openshift-ca\") pod \"7760da1a-a95e-4f3e-9512-08cb4e523cd7\" (UID: \"7760da1a-a95e-4f3e-9512-08cb4e523cd7\") "
Oct 30 19:47:37.636732 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.636657    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2fa1172-31da-4cab-9622-f4adc52c0ca2-maas-api-openshift-ca" pod "b2fa1172-31da-4cab-9622-f4adc52c0ca2" (UID: "b2fa1172-31da-4cab-9622-f4adc52c0ca2"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:47:37.636908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.636840    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7760da1a-a95e-4f3e-9512-08cb4e523cd7-maas-api-openshift-ca" pod "7760da1a-a95e-4f3e-9512-08cb4e523cd7" (UID: "7760da1a-a95e-4f3e-9512-08cb4e523cd7"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:47:37.638679 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.638657    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7760da1a-a95e-4f3e-9512-08cb4e523cd7-kube-api-access-pqtxv" pod "7760da1a-a95e-4f3e-9512-08cb4e523cd7" (UID: "7760da1a-a95e-4f3e-9512-08cb4e523cd7"). InnerVolumeSpecName "kube-api-access-pqtxv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:37.638788 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.638762    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7760da1a-a95e-4f3e-9512-08cb4e523cd7-maas-api-xks-ca" pod "7760da1a-a95e-4f3e-9512-08cb4e523cd7" (UID: "7760da1a-a95e-4f3e-9512-08cb4e523cd7"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:47:37.639212 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.639179    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2fa1172-31da-4cab-9622-f4adc52c0ca2-maas-api-xks-ca" pod "b2fa1172-31da-4cab-9622-f4adc52c0ca2" (UID: "b2fa1172-31da-4cab-9622-f4adc52c0ca2"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:47:37.639503 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.639472    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2fa1172-31da-4cab-9622-f4adc52c0ca2-kube-api-access-4sxt5" pod "b2fa1172-31da-4cab-9622-f4adc52c0ca2" (UID: "b2fa1172-31da-4cab-9622-f4adc52c0ca2"). InnerVolumeSpecName "kube-api-access-4sxt5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:37.737791 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.737712    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7760da1a-a95e-4f3e-9512-08cb4e523cd7-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:37.737791 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.737742    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7760da1a-a95e-4f3e-9512-08cb4e523cd7-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:37.737791 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.737754    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4sxt5\" (UniqueName: \"kubernetes.io/projected/b2fa1172-31da-4cab-9622-f4adc52c0ca2-kube-api-access-4sxt5\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:37.737791 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.737763    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b2fa1172-31da-4cab-9622-f4adc52c0ca2-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:37.737791 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.737772    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b2fa1172-31da-4cab-9622-f4adc52c0ca2-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:37.737791 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:37.737780    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqtxv\" (UniqueName: \"kubernetes.io/projected/7760da1a-a95e-4f3e-9512-08cb4e523cd7-kube-api-access-pqtxv\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:38.089481 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:38.089399    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d04d31e9-592l9"
Oct 30 19:47:38.089481 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:38.089426    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d04d31e9-592l9" event={"ID":"7760da1a-a95e-4f3e-9512-08cb4e523cd7","Type":"ContainerDied","Data":"f287aaabc2a22b33261c7f38d46f7ff1f8ecfc8e20b6bc917aa7a678a89d6d7e"}
Oct 30 19:47:38.090022 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:38.089492    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f287aaabc2a22b33261c7f38d46f7ff1f8ecfc8e20b6bc917aa7a678a89d6d7e"
Oct 30 19:47:38.091090 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:38.091065    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"
Oct 30 19:47:38.091207 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:38.091117    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd" event={"ID":"dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe","Type":"ContainerDied","Data":"23d66735b804293fe6188b28273c173f3a92c1ead051754efc5da507bd5b1ce1"}
Oct 30 19:47:38.091207 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:38.091178    5907 scope.go:122] "RemoveContainer" containerID="77b47da55d91f085d97dca9028eac1585e92bd7f70c3e1fe58186110e915098d"
Oct 30 19:47:38.092726 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:38.092697    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-112c28ee-bmzsd" event={"ID":"b2fa1172-31da-4cab-9622-f4adc52c0ca2","Type":"ContainerDied","Data":"d7e7b1511617aa5326da0431359bb4ff0f070eb344c29f49701b63840b6d401e"}
Oct 30 19:47:38.092822 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:38.092755    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d7e7b1511617aa5326da0431359bb4ff0f070eb344c29f49701b63840b6d401e"
Oct 30 19:47:38.092822 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:38.092759    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-112c28ee-bmzsd"
Oct 30 19:47:38.231918 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:38.231887    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"]
Oct 30 19:47:38.236649 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:38.236620    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-291d35-openshift-default-788c94b755-glrwd"]
Oct 30 19:47:38.288855 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:38.288825    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe" path="/var/lib/kubelet/pods/dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe/volumes"
Oct 30 19:47:41.986892 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:41.986841    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-d797d2-67d78678ff-t4ccl"]
Oct 30 19:47:45.692902 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.692845    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3a667ca0-4bxzw"]
Oct 30 19:47:45.693420 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.693405    5907 cpu_manager.go:414] "containerMap: removing container" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main"
Oct 30 19:47:45.693420 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.693420    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main"
Oct 30 19:47:45.693585 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.693438    5907 cpu_manager.go:414] "containerMap: removing container" podUID="7760da1a-a95e-4f3e-9512-08cb4e523cd7" containerName="revoke-keys"
Oct 30 19:47:45.693585 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.693443    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7760da1a-a95e-4f3e-9512-08cb4e523cd7" containerName="revoke-keys"
Oct 30 19:47:45.693585 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.693450    5907 cpu_manager.go:414] "containerMap: removing container" podUID="b2fa1172-31da-4cab-9622-f4adc52c0ca2" containerName="revoke-keys"
Oct 30 19:47:45.693585 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.693457    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b2fa1172-31da-4cab-9622-f4adc52c0ca2" containerName="revoke-keys"
Oct 30 19:47:45.693585 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.693480    5907 cpu_manager.go:414] "containerMap: removing container" podUID="c960481b-979f-46ab-9edb-78ea467367eb" containerName="revoke-keys"
Oct 30 19:47:45.693585 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.693485    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c960481b-979f-46ab-9edb-78ea467367eb" containerName="revoke-keys"
Oct 30 19:47:45.693585 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.693493    5907 cpu_manager.go:414] "containerMap: removing container" podUID="dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe" containerName="istio-proxy"
Oct 30 19:47:45.693585 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.693498    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe" containerName="istio-proxy"
Oct 30 19:47:45.693585 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.693506    5907 cpu_manager.go:414] "containerMap: removing container" podUID="dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe" containerName="istio-proxy"
Oct 30 19:47:45.693585 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.693527    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe" containerName="istio-proxy"
Oct 30 19:47:45.694142 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.693646    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="c960481b-979f-46ab-9edb-78ea467367eb" containerName="revoke-keys"
Oct 30 19:47:45.694142 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.693654    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe" containerName="istio-proxy"
Oct 30 19:47:45.694142 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.693661    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc09cc6c-b5b2-4108-be0a-35d3c3f03ffe" containerName="istio-proxy"
Oct 30 19:47:45.694142 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.693673    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="7d1837f0-48b2-446c-946c-4007adf95814" containerName="main"
Oct 30 19:47:45.694142 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.693680    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="7760da1a-a95e-4f3e-9512-08cb4e523cd7" containerName="revoke-keys"
Oct 30 19:47:45.694142 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.693686    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="b2fa1172-31da-4cab-9622-f4adc52c0ca2" containerName="revoke-keys"
Oct 30 19:47:45.698938 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.698913    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a667ca0-4bxzw"
Oct 30 19:47:45.710559 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.710534    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1342a88e-rbf8h"]
Oct 30 19:47:45.714671 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.714647    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1342a88e-rbf8h"
Oct 30 19:47:45.718210 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.718185    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3a667ca0-4bxzw"]
Oct 30 19:47:45.720640 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.720619    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:47:45.720762 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.720683    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:47:45.736082 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.736048    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1342a88e-rbf8h"]
Oct 30 19:47:45.858739 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.858702    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dc3f5806-2a1a-4391-b578-f2885bff86f9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3a667ca0-4bxzw\" (UID: \"dc3f5806-2a1a-4391-b578-f2885bff86f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a667ca0-4bxzw"
Oct 30 19:47:45.858971 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.858749    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dc3f5806-2a1a-4391-b578-f2885bff86f9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3a667ca0-4bxzw\" (UID: \"dc3f5806-2a1a-4391-b578-f2885bff86f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a667ca0-4bxzw"
Oct 30 19:47:45.858971 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.858912    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfxvd\" (UniqueName: \"kubernetes.io/projected/dc3f5806-2a1a-4391-b578-f2885bff86f9-kube-api-access-kfxvd\") pod \"maas-api-revoke-sub-3a667ca0-4bxzw\" (UID: \"dc3f5806-2a1a-4391-b578-f2885bff86f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a667ca0-4bxzw"
Oct 30 19:47:45.858971 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.858945    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1583ab53-3fec-484a-92bc-14a5b0a7ac75-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1342a88e-rbf8h\" (UID: \"1583ab53-3fec-484a-92bc-14a5b0a7ac75\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1342a88e-rbf8h"
Oct 30 19:47:45.859111 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.859050    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mwf8\" (UniqueName: \"kubernetes.io/projected/1583ab53-3fec-484a-92bc-14a5b0a7ac75-kube-api-access-7mwf8\") pod \"maas-api-revoke-sub-1342a88e-rbf8h\" (UID: \"1583ab53-3fec-484a-92bc-14a5b0a7ac75\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1342a88e-rbf8h"
Oct 30 19:47:45.859111 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.859094    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1583ab53-3fec-484a-92bc-14a5b0a7ac75-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1342a88e-rbf8h\" (UID: \"1583ab53-3fec-484a-92bc-14a5b0a7ac75\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1342a88e-rbf8h"
Oct 30 19:47:45.970301 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.970205    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mwf8\" (UniqueName: \"kubernetes.io/projected/1583ab53-3fec-484a-92bc-14a5b0a7ac75-kube-api-access-7mwf8\") pod \"maas-api-revoke-sub-1342a88e-rbf8h\" (UID: \"1583ab53-3fec-484a-92bc-14a5b0a7ac75\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1342a88e-rbf8h"
Oct 30 19:47:45.970977 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.970949    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1583ab53-3fec-484a-92bc-14a5b0a7ac75-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1342a88e-rbf8h\" (UID: \"1583ab53-3fec-484a-92bc-14a5b0a7ac75\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1342a88e-rbf8h"
Oct 30 19:47:45.971938 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.971913    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dc3f5806-2a1a-4391-b578-f2885bff86f9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3a667ca0-4bxzw\" (UID: \"dc3f5806-2a1a-4391-b578-f2885bff86f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a667ca0-4bxzw"
Oct 30 19:47:45.972262 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.972239    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dc3f5806-2a1a-4391-b578-f2885bff86f9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3a667ca0-4bxzw\" (UID: \"dc3f5806-2a1a-4391-b578-f2885bff86f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a667ca0-4bxzw"
Oct 30 19:47:45.972443 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.972428    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfxvd\" (UniqueName: \"kubernetes.io/projected/dc3f5806-2a1a-4391-b578-f2885bff86f9-kube-api-access-kfxvd\") pod \"maas-api-revoke-sub-3a667ca0-4bxzw\" (UID: \"dc3f5806-2a1a-4391-b578-f2885bff86f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a667ca0-4bxzw"
Oct 30 19:47:45.972566 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.972552    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1583ab53-3fec-484a-92bc-14a5b0a7ac75-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1342a88e-rbf8h\" (UID: \"1583ab53-3fec-484a-92bc-14a5b0a7ac75\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1342a88e-rbf8h"
Oct 30 19:47:45.973556 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.973427    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1583ab53-3fec-484a-92bc-14a5b0a7ac75-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1342a88e-rbf8h\" (UID: \"1583ab53-3fec-484a-92bc-14a5b0a7ac75\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1342a88e-rbf8h"
Oct 30 19:47:45.974710 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.974688    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dc3f5806-2a1a-4391-b578-f2885bff86f9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3a667ca0-4bxzw\" (UID: \"dc3f5806-2a1a-4391-b578-f2885bff86f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a667ca0-4bxzw"
Oct 30 19:47:45.978055 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:45.978034    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1583ab53-3fec-484a-92bc-14a5b0a7ac75-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1342a88e-rbf8h\" (UID: \"1583ab53-3fec-484a-92bc-14a5b0a7ac75\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1342a88e-rbf8h"
Oct 30 19:47:46.016721 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:46.005240    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dc3f5806-2a1a-4391-b578-f2885bff86f9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3a667ca0-4bxzw\" (UID: \"dc3f5806-2a1a-4391-b578-f2885bff86f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a667ca0-4bxzw"
Oct 30 19:47:46.021792 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:46.021729    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfxvd\" (UniqueName: \"kubernetes.io/projected/dc3f5806-2a1a-4391-b578-f2885bff86f9-kube-api-access-kfxvd\") pod \"maas-api-revoke-sub-3a667ca0-4bxzw\" (UID: \"dc3f5806-2a1a-4391-b578-f2885bff86f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a667ca0-4bxzw"
Oct 30 19:47:46.030884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:46.022828    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mwf8\" (UniqueName: \"kubernetes.io/projected/1583ab53-3fec-484a-92bc-14a5b0a7ac75-kube-api-access-7mwf8\") pod \"maas-api-revoke-sub-1342a88e-rbf8h\" (UID: \"1583ab53-3fec-484a-92bc-14a5b0a7ac75\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1342a88e-rbf8h"
Oct 30 19:47:46.034888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:46.031072    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1342a88e-rbf8h"
Oct 30 19:47:46.314100 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:46.314010    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a667ca0-4bxzw"
Oct 30 19:47:46.400163 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:46.400065    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1342a88e-rbf8h"]
Oct 30 19:47:46.449667 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:46.449628    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f913c6-8g4fc"]
Oct 30 19:47:46.453593 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:46.453563    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f913c6-8g4fc"]
Oct 30 19:47:46.627198 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:46.627169    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3a667ca0-4bxzw"]
Oct 30 19:47:46.627715 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:47:46.627690    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc3f5806_2a1a_4391_b578_f2885bff86f9.slice/crio-92fa27e0b0604a90cc237e9adcd8a44e21e00f3dfbe7bb191f73ff08fc84f2c2 WatchSource:0}: Error finding container 92fa27e0b0604a90cc237e9adcd8a44e21e00f3dfbe7bb191f73ff08fc84f2c2: Status 404 returned error can't find the container with id 92fa27e0b0604a90cc237e9adcd8a44e21e00f3dfbe7bb191f73ff08fc84f2c2
Oct 30 19:47:46.987955 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:46.987917    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-d797d2-67d78678ff-t4ccl" podUID="55e11ef7-bf87-4002-93e8-46af5da4c8f0" containerName="payload-pre-processing" containerID="cri-o://39096ffc723273012185e392683bfad7f6cd989318acbe01c9cb6377615948b9" gracePeriod=25
Oct 30 19:47:47.221500 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:47.221408    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1342a88e-rbf8h" event={"ID":"1583ab53-3fec-484a-92bc-14a5b0a7ac75","Type":"ContainerStarted","Data":"a965ee70eba673c8d0e1d65930079ae9d53cc9e0f76b983aed4f78556ab1ab99"}
Oct 30 19:47:47.221500 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:47.221480    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1342a88e-rbf8h" event={"ID":"1583ab53-3fec-484a-92bc-14a5b0a7ac75","Type":"ContainerStarted","Data":"341bb1cf2df52837479e5bc8fcf3bc478caaff7e0afb62990d60f647fee639ab"}
Oct 30 19:47:47.223346 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:47.223315    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a667ca0-4bxzw" event={"ID":"dc3f5806-2a1a-4391-b578-f2885bff86f9","Type":"ContainerStarted","Data":"8687fbdc0eb1827e2854f1c4e917034b7e671a442938129d8540e5f0b256f209"}
Oct 30 19:47:47.223462 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:47.223364    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a667ca0-4bxzw" event={"ID":"dc3f5806-2a1a-4391-b578-f2885bff86f9","Type":"ContainerStarted","Data":"92fa27e0b0604a90cc237e9adcd8a44e21e00f3dfbe7bb191f73ff08fc84f2c2"}
Oct 30 19:47:47.270037 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:47.269981    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1342a88e-rbf8h" podStartSLOduration=2.26996561 podStartE2EDuration="2.26996561s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:47:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:47.266057836 +0000 UTC m=+1579.108989774" watchObservedRunningTime="2026-10-30 19:47:47.26996561 +0000 UTC m=+1579.112897582"
Oct 30 19:47:47.316684 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:47.316630    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a667ca0-4bxzw" podStartSLOduration=2.316615396 podStartE2EDuration="2.316615396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:47:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:47.313198116 +0000 UTC m=+1579.156130056" watchObservedRunningTime="2026-10-30 19:47:47.316615396 +0000 UTC m=+1579.159547326"
Oct 30 19:47:47.500678 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:47:47.500593    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:47:48.232051 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:48.232017    5907 generic.go:356] "Generic (PLEG): container finished" podID="1583ab53-3fec-484a-92bc-14a5b0a7ac75" containerID="a965ee70eba673c8d0e1d65930079ae9d53cc9e0f76b983aed4f78556ab1ab99" exitCode=0
Oct 30 19:47:48.232503 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:48.232106    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1342a88e-rbf8h" event={"ID":"1583ab53-3fec-484a-92bc-14a5b0a7ac75","Type":"ContainerDied","Data":"a965ee70eba673c8d0e1d65930079ae9d53cc9e0f76b983aed4f78556ab1ab99"}
Oct 30 19:47:48.233781 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:48.233757    5907 generic.go:356] "Generic (PLEG): container finished" podID="dc3f5806-2a1a-4391-b578-f2885bff86f9" containerID="8687fbdc0eb1827e2854f1c4e917034b7e671a442938129d8540e5f0b256f209" exitCode=0
Oct 30 19:47:48.233921 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:48.233841    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a667ca0-4bxzw" event={"ID":"dc3f5806-2a1a-4391-b578-f2885bff86f9","Type":"ContainerDied","Data":"8687fbdc0eb1827e2854f1c4e917034b7e671a442938129d8540e5f0b256f209"}
Oct 30 19:47:48.287462 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:48.287426    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c0fc0b73-412c-46de-bac5-69f35c8c229b" path="/var/lib/kubelet/pods/c0fc0b73-412c-46de-bac5-69f35c8c229b/volumes"
Oct 30 19:47:49.413944 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:49.413912    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1342a88e-rbf8h"
Oct 30 19:47:49.417280 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:49.417260    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a667ca0-4bxzw"
Oct 30 19:47:49.542315 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:49.542232    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc3f5806-2a1a-4391-b578-f2885bff86f9-kube-api-access-kfxvd\" (UniqueName: \"kubernetes.io/projected/dc3f5806-2a1a-4391-b578-f2885bff86f9-kube-api-access-kfxvd\") pod \"dc3f5806-2a1a-4391-b578-f2885bff86f9\" (UID: \"dc3f5806-2a1a-4391-b578-f2885bff86f9\") "
Oct 30 19:47:49.542315 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:49.542278    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1583ab53-3fec-484a-92bc-14a5b0a7ac75-kube-api-access-7mwf8\" (UniqueName: \"kubernetes.io/projected/1583ab53-3fec-484a-92bc-14a5b0a7ac75-kube-api-access-7mwf8\") pod \"1583ab53-3fec-484a-92bc-14a5b0a7ac75\" (UID: \"1583ab53-3fec-484a-92bc-14a5b0a7ac75\") "
Oct 30 19:47:49.542593 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:49.542354    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dc3f5806-2a1a-4391-b578-f2885bff86f9-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dc3f5806-2a1a-4391-b578-f2885bff86f9-maas-api-openshift-ca\") pod \"dc3f5806-2a1a-4391-b578-f2885bff86f9\" (UID: \"dc3f5806-2a1a-4391-b578-f2885bff86f9\") "
Oct 30 19:47:49.542593 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:49.542395    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dc3f5806-2a1a-4391-b578-f2885bff86f9-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dc3f5806-2a1a-4391-b578-f2885bff86f9-maas-api-xks-ca\") pod \"dc3f5806-2a1a-4391-b578-f2885bff86f9\" (UID: \"dc3f5806-2a1a-4391-b578-f2885bff86f9\") "
Oct 30 19:47:49.542593 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:49.542519    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1583ab53-3fec-484a-92bc-14a5b0a7ac75-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1583ab53-3fec-484a-92bc-14a5b0a7ac75-maas-api-openshift-ca\") pod \"1583ab53-3fec-484a-92bc-14a5b0a7ac75\" (UID: \"1583ab53-3fec-484a-92bc-14a5b0a7ac75\") "
Oct 30 19:47:49.542754 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:49.542594    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1583ab53-3fec-484a-92bc-14a5b0a7ac75-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1583ab53-3fec-484a-92bc-14a5b0a7ac75-maas-api-xks-ca\") pod \"1583ab53-3fec-484a-92bc-14a5b0a7ac75\" (UID: \"1583ab53-3fec-484a-92bc-14a5b0a7ac75\") "
Oct 30 19:47:49.543031 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:49.542990    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1583ab53-3fec-484a-92bc-14a5b0a7ac75-maas-api-openshift-ca" pod "1583ab53-3fec-484a-92bc-14a5b0a7ac75" (UID: "1583ab53-3fec-484a-92bc-14a5b0a7ac75"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:47:49.543140 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:49.543091    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc3f5806-2a1a-4391-b578-f2885bff86f9-maas-api-openshift-ca" pod "dc3f5806-2a1a-4391-b578-f2885bff86f9" (UID: "dc3f5806-2a1a-4391-b578-f2885bff86f9"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:47:49.543251 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:49.543231    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1583ab53-3fec-484a-92bc-14a5b0a7ac75-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:49.543320 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:49.543258    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dc3f5806-2a1a-4391-b578-f2885bff86f9-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:49.545638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:49.545366    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc3f5806-2a1a-4391-b578-f2885bff86f9-kube-api-access-kfxvd" pod "dc3f5806-2a1a-4391-b578-f2885bff86f9" (UID: "dc3f5806-2a1a-4391-b578-f2885bff86f9"). InnerVolumeSpecName "kube-api-access-kfxvd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:49.546011 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:49.545945    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc3f5806-2a1a-4391-b578-f2885bff86f9-maas-api-xks-ca" pod "dc3f5806-2a1a-4391-b578-f2885bff86f9" (UID: "dc3f5806-2a1a-4391-b578-f2885bff86f9"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:47:49.546011 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:49.545978    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1583ab53-3fec-484a-92bc-14a5b0a7ac75-kube-api-access-7mwf8" pod "1583ab53-3fec-484a-92bc-14a5b0a7ac75" (UID: "1583ab53-3fec-484a-92bc-14a5b0a7ac75"). InnerVolumeSpecName "kube-api-access-7mwf8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:49.546169 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:49.546009    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1583ab53-3fec-484a-92bc-14a5b0a7ac75-maas-api-xks-ca" pod "1583ab53-3fec-484a-92bc-14a5b0a7ac75" (UID: "1583ab53-3fec-484a-92bc-14a5b0a7ac75"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:47:49.644967 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:49.644932    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1583ab53-3fec-484a-92bc-14a5b0a7ac75-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:49.644967 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:49.644961    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kfxvd\" (UniqueName: \"kubernetes.io/projected/dc3f5806-2a1a-4391-b578-f2885bff86f9-kube-api-access-kfxvd\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:49.644967 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:49.644972    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mwf8\" (UniqueName: \"kubernetes.io/projected/1583ab53-3fec-484a-92bc-14a5b0a7ac75-kube-api-access-7mwf8\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:49.645223 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:49.644981    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dc3f5806-2a1a-4391-b578-f2885bff86f9-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:47:50.249284 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:50.249255    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1342a88e-rbf8h"
Oct 30 19:47:50.249475 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:50.249258    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1342a88e-rbf8h" event={"ID":"1583ab53-3fec-484a-92bc-14a5b0a7ac75","Type":"ContainerDied","Data":"341bb1cf2df52837479e5bc8fcf3bc478caaff7e0afb62990d60f647fee639ab"}
Oct 30 19:47:50.249475 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:50.249369    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="341bb1cf2df52837479e5bc8fcf3bc478caaff7e0afb62990d60f647fee639ab"
Oct 30 19:47:50.250621 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:50.250604    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a667ca0-4bxzw"
Oct 30 19:47:50.250726 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:50.250655    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3a667ca0-4bxzw" event={"ID":"dc3f5806-2a1a-4391-b578-f2885bff86f9","Type":"ContainerDied","Data":"92fa27e0b0604a90cc237e9adcd8a44e21e00f3dfbe7bb191f73ff08fc84f2c2"}
Oct 30 19:47:50.250726 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:50.250701    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="92fa27e0b0604a90cc237e9adcd8a44e21e00f3dfbe7bb191f73ff08fc84f2c2"
Oct 30 19:47:51.986002 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:51.985964    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-d797d2-67d78678ff-t4ccl" podUID="55e11ef7-bf87-4002-93e8-46af5da4c8f0" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:47:52.130888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:52.117548    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:47:56.623854 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:56.623802    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"]
Oct 30 19:47:56.624378 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:56.624100    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5" podUID="0fbf8d00-57a5-49ba-855a-4accc2f1bb05" containerName="istio-proxy" containerID="cri-o://fbb146431899285ec8a1f0d33e1164a44016cd50160aa548a1d66672cd7b50b7" gracePeriod=30
Oct 30 19:47:59.994808 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:59.994764    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d20ed19a-9btfj"]
Oct 30 19:47:59.995595 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:59.995573    5907 cpu_manager.go:414] "containerMap: removing container" podUID="1583ab53-3fec-484a-92bc-14a5b0a7ac75" containerName="revoke-keys"
Oct 30 19:47:59.995702 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:59.995596    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1583ab53-3fec-484a-92bc-14a5b0a7ac75" containerName="revoke-keys"
Oct 30 19:47:59.995702 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:59.995623    5907 cpu_manager.go:414] "containerMap: removing container" podUID="dc3f5806-2a1a-4391-b578-f2885bff86f9" containerName="revoke-keys"
Oct 30 19:47:59.995702 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:59.995630    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc3f5806-2a1a-4391-b578-f2885bff86f9" containerName="revoke-keys"
Oct 30 19:47:59.995880 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:59.995760    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="1583ab53-3fec-484a-92bc-14a5b0a7ac75" containerName="revoke-keys"
Oct 30 19:47:59.995880 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:59.995777    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc3f5806-2a1a-4391-b578-f2885bff86f9" containerName="revoke-keys"
Oct 30 19:47:59.999723 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:47:59.999702    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d20ed19a-9btfj"
Oct 30 19:48:00.021250 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:00.021224    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d20ed19a-9btfj"]
Oct 30 19:48:00.029884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:00.029834    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:48:00.029884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:00.029874    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:48:00.085072 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:00.085039    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-956np\" (UniqueName: \"kubernetes.io/projected/fb5c110f-658c-46af-93b1-2122cc7b37b0-kube-api-access-956np\") pod \"maas-api-revoke-sub-d20ed19a-9btfj\" (UID: \"fb5c110f-658c-46af-93b1-2122cc7b37b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d20ed19a-9btfj"
Oct 30 19:48:00.085254 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:00.085186    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fb5c110f-658c-46af-93b1-2122cc7b37b0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d20ed19a-9btfj\" (UID: \"fb5c110f-658c-46af-93b1-2122cc7b37b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d20ed19a-9btfj"
Oct 30 19:48:00.085254 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:00.085236    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fb5c110f-658c-46af-93b1-2122cc7b37b0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d20ed19a-9btfj\" (UID: \"fb5c110f-658c-46af-93b1-2122cc7b37b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d20ed19a-9btfj"
Oct 30 19:48:00.186294 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:00.186258    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-956np\" (UniqueName: \"kubernetes.io/projected/fb5c110f-658c-46af-93b1-2122cc7b37b0-kube-api-access-956np\") pod \"maas-api-revoke-sub-d20ed19a-9btfj\" (UID: \"fb5c110f-658c-46af-93b1-2122cc7b37b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d20ed19a-9btfj"
Oct 30 19:48:00.186499 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:00.186351    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fb5c110f-658c-46af-93b1-2122cc7b37b0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d20ed19a-9btfj\" (UID: \"fb5c110f-658c-46af-93b1-2122cc7b37b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d20ed19a-9btfj"
Oct 30 19:48:00.186499 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:00.186386    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fb5c110f-658c-46af-93b1-2122cc7b37b0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d20ed19a-9btfj\" (UID: \"fb5c110f-658c-46af-93b1-2122cc7b37b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d20ed19a-9btfj"
Oct 30 19:48:00.187039 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:00.187017    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fb5c110f-658c-46af-93b1-2122cc7b37b0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d20ed19a-9btfj\" (UID: \"fb5c110f-658c-46af-93b1-2122cc7b37b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d20ed19a-9btfj"
Oct 30 19:48:00.188913 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:00.188885    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fb5c110f-658c-46af-93b1-2122cc7b37b0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d20ed19a-9btfj\" (UID: \"fb5c110f-658c-46af-93b1-2122cc7b37b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d20ed19a-9btfj"
Oct 30 19:48:00.236638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:00.236595    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-956np\" (UniqueName: \"kubernetes.io/projected/fb5c110f-658c-46af-93b1-2122cc7b37b0-kube-api-access-956np\") pod \"maas-api-revoke-sub-d20ed19a-9btfj\" (UID: \"fb5c110f-658c-46af-93b1-2122cc7b37b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d20ed19a-9btfj"
Oct 30 19:48:00.313526 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:00.313441    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d20ed19a-9btfj"
Oct 30 19:48:00.633102 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:00.633072    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d20ed19a-9btfj"]
Oct 30 19:48:00.633602 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:48:00.633574    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb5c110f_658c_46af_93b1_2122cc7b37b0.slice/crio-2245cbd95dbdceb5ae811b2b3ed55278eb2f6cecec4687b0502c28a23a6838ed WatchSource:0}: Error finding container 2245cbd95dbdceb5ae811b2b3ed55278eb2f6cecec4687b0502c28a23a6838ed: Status 404 returned error can't find the container with id 2245cbd95dbdceb5ae811b2b3ed55278eb2f6cecec4687b0502c28a23a6838ed
Oct 30 19:48:01.338523 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:01.338418    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d20ed19a-9btfj" event={"ID":"fb5c110f-658c-46af-93b1-2122cc7b37b0","Type":"ContainerStarted","Data":"7b2f2db1917cff6c20bd4af02b1c8882c193ef198065759b4ee69a0b8f4e0b28"}
Oct 30 19:48:01.338523 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:01.338485    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d20ed19a-9btfj" event={"ID":"fb5c110f-658c-46af-93b1-2122cc7b37b0","Type":"ContainerStarted","Data":"2245cbd95dbdceb5ae811b2b3ed55278eb2f6cecec4687b0502c28a23a6838ed"}
Oct 30 19:48:01.394582 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:01.394520    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d20ed19a-9btfj" podStartSLOduration=2.394503513 podStartE2EDuration="2.394503513s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:47:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:48:01.392559411 +0000 UTC m=+1593.235491349" watchObservedRunningTime="2026-10-30 19:48:01.394503513 +0000 UTC m=+1593.237435453"
Oct 30 19:48:01.926712 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:01.926682    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"
Oct 30 19:48:01.985978 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:01.985922    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-d797d2-67d78678ff-t4ccl" podUID="55e11ef7-bf87-4002-93e8-46af5da4c8f0" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:48:02.011532 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.011496    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-istio-envoy\") pod \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") "
Oct 30 19:48:02.011728 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.011583    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-istio-podinfo\") pod \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") "
Oct 30 19:48:02.011728 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.011654    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-istiod-ca-cert\") pod \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") "
Oct 30 19:48:02.011855 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.011778    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-workload-socket\") pod \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") "
Oct 30 19:48:02.011855 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.011836    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-workload-certs\") pod \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") "
Oct 30 19:48:02.011995 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.011925    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-istio-data\") pod \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") "
Oct 30 19:48:02.011995 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.011990    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-kube-api-access-cpjw4\" (UniqueName: \"kubernetes.io/projected/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-kube-api-access-cpjw4\") pod \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") "
Oct 30 19:48:02.012121 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.012022    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-istio-token\" (UniqueName: \"kubernetes.io/projected/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-istio-token\") pod \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") "
Oct 30 19:48:02.012121 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.012077    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-istiod-ca-cert" pod "0fbf8d00-57a5-49ba-855a-4accc2f1bb05" (UID: "0fbf8d00-57a5-49ba-855a-4accc2f1bb05"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:48:02.012225 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.012144    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-credential-socket\") pod \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\" (UID: \"0fbf8d00-57a5-49ba-855a-4accc2f1bb05\") "
Oct 30 19:48:02.012719 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.012694    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:02.012719 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.012709    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-istio-data" pod "0fbf8d00-57a5-49ba-855a-4accc2f1bb05" (UID: "0fbf8d00-57a5-49ba-855a-4accc2f1bb05"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:02.012991 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.012793    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-workload-certs" pod "0fbf8d00-57a5-49ba-855a-4accc2f1bb05" (UID: "0fbf8d00-57a5-49ba-855a-4accc2f1bb05"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:02.012991 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.012846    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-workload-socket" pod "0fbf8d00-57a5-49ba-855a-4accc2f1bb05" (UID: "0fbf8d00-57a5-49ba-855a-4accc2f1bb05"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:02.013126 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.012954    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-credential-socket" pod "0fbf8d00-57a5-49ba-855a-4accc2f1bb05" (UID: "0fbf8d00-57a5-49ba-855a-4accc2f1bb05"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:02.014590 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.014550    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-istio-envoy" pod "0fbf8d00-57a5-49ba-855a-4accc2f1bb05" (UID: "0fbf8d00-57a5-49ba-855a-4accc2f1bb05"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:02.014779 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.014757    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-kube-api-access-cpjw4" pod "0fbf8d00-57a5-49ba-855a-4accc2f1bb05" (UID: "0fbf8d00-57a5-49ba-855a-4accc2f1bb05"). InnerVolumeSpecName "kube-api-access-cpjw4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:02.014924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.014817    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-istio-podinfo" pod "0fbf8d00-57a5-49ba-855a-4accc2f1bb05" (UID: "0fbf8d00-57a5-49ba-855a-4accc2f1bb05"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:48:02.015048 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.015021    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-istio-token" pod "0fbf8d00-57a5-49ba-855a-4accc2f1bb05" (UID: "0fbf8d00-57a5-49ba-855a-4accc2f1bb05"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:02.113981 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.113943    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:02.113981 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.113975    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:02.113981 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.113984    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:02.114241 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.113992    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpjw4\" (UniqueName: \"kubernetes.io/projected/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-kube-api-access-cpjw4\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:02.114241 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.114001    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:02.114241 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.114009    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:02.114241 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.114017    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:02.114241 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.114025    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fbf8d00-57a5-49ba-855a-4accc2f1bb05-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:02.346106 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.346073    5907 generic.go:356] "Generic (PLEG): container finished" podID="0fbf8d00-57a5-49ba-855a-4accc2f1bb05" containerID="fbb146431899285ec8a1f0d33e1164a44016cd50160aa548a1d66672cd7b50b7" exitCode=0
Oct 30 19:48:02.346580 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.346127    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"
Oct 30 19:48:02.346580 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.346159    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5" event={"ID":"0fbf8d00-57a5-49ba-855a-4accc2f1bb05","Type":"ContainerDied","Data":"fbb146431899285ec8a1f0d33e1164a44016cd50160aa548a1d66672cd7b50b7"}
Oct 30 19:48:02.346580 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.346217    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5" event={"ID":"0fbf8d00-57a5-49ba-855a-4accc2f1bb05","Type":"ContainerDied","Data":"a0f42be8d64860d5ed99b63af2bf44d6b93a01f52580facba2036ddcf3ec569d"}
Oct 30 19:48:02.346580 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.346244    5907 scope.go:122] "RemoveContainer" containerID="fbb146431899285ec8a1f0d33e1164a44016cd50160aa548a1d66672cd7b50b7"
Oct 30 19:48:02.348229 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.348200    5907 generic.go:356] "Generic (PLEG): container finished" podID="fb5c110f-658c-46af-93b1-2122cc7b37b0" containerID="7b2f2db1917cff6c20bd4af02b1c8882c193ef198065759b4ee69a0b8f4e0b28" exitCode=0
Oct 30 19:48:02.348368 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.348303    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d20ed19a-9btfj" event={"ID":"fb5c110f-658c-46af-93b1-2122cc7b37b0","Type":"ContainerDied","Data":"7b2f2db1917cff6c20bd4af02b1c8882c193ef198065759b4ee69a0b8f4e0b28"}
Oct 30 19:48:02.356852 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.356834    5907 scope.go:122] "RemoveContainer" containerID="fbb146431899285ec8a1f0d33e1164a44016cd50160aa548a1d66672cd7b50b7"
Oct 30 19:48:02.357164 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:48:02.357146    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbb146431899285ec8a1f0d33e1164a44016cd50160aa548a1d66672cd7b50b7\": container with ID starting with fbb146431899285ec8a1f0d33e1164a44016cd50160aa548a1d66672cd7b50b7 not found: ID does not exist" containerID="fbb146431899285ec8a1f0d33e1164a44016cd50160aa548a1d66672cd7b50b7"
Oct 30 19:48:02.357237 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.357177    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbb146431899285ec8a1f0d33e1164a44016cd50160aa548a1d66672cd7b50b7"} err="failed to get container status \"fbb146431899285ec8a1f0d33e1164a44016cd50160aa548a1d66672cd7b50b7\": rpc error: code = NotFound desc = could not find container \"fbb146431899285ec8a1f0d33e1164a44016cd50160aa548a1d66672cd7b50b7\": container with ID starting with fbb146431899285ec8a1f0d33e1164a44016cd50160aa548a1d66672cd7b50b7 not found: ID does not exist"
Oct 30 19:48:02.410360 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.410329    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"]
Oct 30 19:48:02.423439 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:02.423412    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-f913c6-openshift-default-85664fdcfc-pzsn5"]
Oct 30 19:48:03.506643 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:03.506620    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d20ed19a-9btfj"
Oct 30 19:48:03.629889 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:03.629843    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fb5c110f-658c-46af-93b1-2122cc7b37b0-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fb5c110f-658c-46af-93b1-2122cc7b37b0-maas-api-openshift-ca\") pod \"fb5c110f-658c-46af-93b1-2122cc7b37b0\" (UID: \"fb5c110f-658c-46af-93b1-2122cc7b37b0\") "
Oct 30 19:48:03.630072 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:03.629998    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fb5c110f-658c-46af-93b1-2122cc7b37b0-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fb5c110f-658c-46af-93b1-2122cc7b37b0-maas-api-xks-ca\") pod \"fb5c110f-658c-46af-93b1-2122cc7b37b0\" (UID: \"fb5c110f-658c-46af-93b1-2122cc7b37b0\") "
Oct 30 19:48:03.630158 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:03.630087    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fb5c110f-658c-46af-93b1-2122cc7b37b0-kube-api-access-956np\" (UniqueName: \"kubernetes.io/projected/fb5c110f-658c-46af-93b1-2122cc7b37b0-kube-api-access-956np\") pod \"fb5c110f-658c-46af-93b1-2122cc7b37b0\" (UID: \"fb5c110f-658c-46af-93b1-2122cc7b37b0\") "
Oct 30 19:48:03.630259 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:03.630232    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb5c110f-658c-46af-93b1-2122cc7b37b0-maas-api-openshift-ca" pod "fb5c110f-658c-46af-93b1-2122cc7b37b0" (UID: "fb5c110f-658c-46af-93b1-2122cc7b37b0"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:48:03.630532 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:03.630514    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fb5c110f-658c-46af-93b1-2122cc7b37b0-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:03.633275 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:03.633250    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb5c110f-658c-46af-93b1-2122cc7b37b0-kube-api-access-956np" pod "fb5c110f-658c-46af-93b1-2122cc7b37b0" (UID: "fb5c110f-658c-46af-93b1-2122cc7b37b0"). InnerVolumeSpecName "kube-api-access-956np". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:03.633540 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:03.633464    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb5c110f-658c-46af-93b1-2122cc7b37b0-maas-api-xks-ca" pod "fb5c110f-658c-46af-93b1-2122cc7b37b0" (UID: "fb5c110f-658c-46af-93b1-2122cc7b37b0"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:48:03.731468 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:03.731433    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fb5c110f-658c-46af-93b1-2122cc7b37b0-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:03.731639 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:03.731483    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-956np\" (UniqueName: \"kubernetes.io/projected/fb5c110f-658c-46af-93b1-2122cc7b37b0-kube-api-access-956np\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:04.287668 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:04.287633    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0fbf8d00-57a5-49ba-855a-4accc2f1bb05" path="/var/lib/kubelet/pods/0fbf8d00-57a5-49ba-855a-4accc2f1bb05/volumes"
Oct 30 19:48:04.365108 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:04.365072    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d20ed19a-9btfj"
Oct 30 19:48:04.365302 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:04.365110    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d20ed19a-9btfj" event={"ID":"fb5c110f-658c-46af-93b1-2122cc7b37b0","Type":"ContainerDied","Data":"2245cbd95dbdceb5ae811b2b3ed55278eb2f6cecec4687b0502c28a23a6838ed"}
Oct 30 19:48:04.365302 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:04.365157    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2245cbd95dbdceb5ae811b2b3ed55278eb2f6cecec4687b0502c28a23a6838ed"
Oct 30 19:48:06.078923 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:06.078884    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-d797d2-cplms"]
Oct 30 19:48:06.091976 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:06.091944    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-d797d2-cplms"]
Oct 30 19:48:06.288245 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:06.288211    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c960481b-979f-46ab-9edb-78ea467367eb" path="/var/lib/kubelet/pods/c960481b-979f-46ab-9edb-78ea467367eb/volumes"
Oct 30 19:48:07.212232 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:07.212192    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:48:07.255788 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:07.255710    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-d797d2-67d78678ff-t4ccl"
Oct 30 19:48:07.374921 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:07.374848    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/55e11ef7-bf87-4002-93e8-46af5da4c8f0-kube-api-access-fgh9j\" (UniqueName: \"kubernetes.io/projected/55e11ef7-bf87-4002-93e8-46af5da4c8f0-kube-api-access-fgh9j\") pod \"55e11ef7-bf87-4002-93e8-46af5da4c8f0\" (UID: \"55e11ef7-bf87-4002-93e8-46af5da4c8f0\") "
Oct 30 19:48:07.374921 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:07.374918    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/55e11ef7-bf87-4002-93e8-46af5da4c8f0-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55e11ef7-bf87-4002-93e8-46af5da4c8f0-plugins-config-volume\") pod \"55e11ef7-bf87-4002-93e8-46af5da4c8f0\" (UID: \"55e11ef7-bf87-4002-93e8-46af5da4c8f0\") "
Oct 30 19:48:07.375303 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:07.375272    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55e11ef7-bf87-4002-93e8-46af5da4c8f0-plugins-config-volume" pod "55e11ef7-bf87-4002-93e8-46af5da4c8f0" (UID: "55e11ef7-bf87-4002-93e8-46af5da4c8f0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:48:07.377233 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:07.377206    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55e11ef7-bf87-4002-93e8-46af5da4c8f0-kube-api-access-fgh9j" pod "55e11ef7-bf87-4002-93e8-46af5da4c8f0" (UID: "55e11ef7-bf87-4002-93e8-46af5da4c8f0"). InnerVolumeSpecName "kube-api-access-fgh9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:07.388034 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:07.388008    5907 generic.go:356] "Generic (PLEG): container finished" podID="55e11ef7-bf87-4002-93e8-46af5da4c8f0" containerID="39096ffc723273012185e392683bfad7f6cd989318acbe01c9cb6377615948b9" exitCode=0
Oct 30 19:48:07.388173 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:07.388064    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-d797d2-67d78678ff-t4ccl"
Oct 30 19:48:07.388173 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:07.388095    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-d797d2-67d78678ff-t4ccl" event={"ID":"55e11ef7-bf87-4002-93e8-46af5da4c8f0","Type":"ContainerDied","Data":"39096ffc723273012185e392683bfad7f6cd989318acbe01c9cb6377615948b9"}
Oct 30 19:48:07.388173 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:07.388154    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-d797d2-67d78678ff-t4ccl" event={"ID":"55e11ef7-bf87-4002-93e8-46af5da4c8f0","Type":"ContainerDied","Data":"5f4ee057a5726af04e5c1b1bd083b52fcbdbc449b7e89281e0b51596b79ce4c1"}
Oct 30 19:48:07.388329 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:07.388177    5907 scope.go:122] "RemoveContainer" containerID="39096ffc723273012185e392683bfad7f6cd989318acbe01c9cb6377615948b9"
Oct 30 19:48:07.398607 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:07.398590    5907 scope.go:122] "RemoveContainer" containerID="39096ffc723273012185e392683bfad7f6cd989318acbe01c9cb6377615948b9"
Oct 30 19:48:07.398897 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:48:07.398879    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39096ffc723273012185e392683bfad7f6cd989318acbe01c9cb6377615948b9\": container with ID starting with 39096ffc723273012185e392683bfad7f6cd989318acbe01c9cb6377615948b9 not found: ID does not exist" containerID="39096ffc723273012185e392683bfad7f6cd989318acbe01c9cb6377615948b9"
Oct 30 19:48:07.398949 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:07.398906    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39096ffc723273012185e392683bfad7f6cd989318acbe01c9cb6377615948b9"} err="failed to get container status \"39096ffc723273012185e392683bfad7f6cd989318acbe01c9cb6377615948b9\": rpc error: code = NotFound desc = could not find container \"39096ffc723273012185e392683bfad7f6cd989318acbe01c9cb6377615948b9\": container with ID starting with 39096ffc723273012185e392683bfad7f6cd989318acbe01c9cb6377615948b9 not found: ID does not exist"
Oct 30 19:48:07.443652 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:07.443608    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-d797d2-67d78678ff-t4ccl"]
Oct 30 19:48:07.448099 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:07.448073    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-d797d2-67d78678ff-t4ccl"]
Oct 30 19:48:07.476152 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:07.476112    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgh9j\" (UniqueName: \"kubernetes.io/projected/55e11ef7-bf87-4002-93e8-46af5da4c8f0-kube-api-access-fgh9j\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:07.476152 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:07.476144    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55e11ef7-bf87-4002-93e8-46af5da4c8f0-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:08.290309 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:08.290270    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="55e11ef7-bf87-4002-93e8-46af5da4c8f0" path="/var/lib/kubelet/pods/55e11ef7-bf87-4002-93e8-46af5da4c8f0/volumes"
Oct 30 19:48:10.553408 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:10.553363    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w5-3c89af-csp6c"]
Oct 30 19:48:10.553978 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:10.553963    5907 cpu_manager.go:414] "containerMap: removing container" podUID="fb5c110f-658c-46af-93b1-2122cc7b37b0" containerName="revoke-keys"
Oct 30 19:48:10.554036 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:10.553980    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fb5c110f-658c-46af-93b1-2122cc7b37b0" containerName="revoke-keys"
Oct 30 19:48:10.554036 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:10.554004    5907 cpu_manager.go:414] "containerMap: removing container" podUID="55e11ef7-bf87-4002-93e8-46af5da4c8f0" containerName="payload-pre-processing"
Oct 30 19:48:10.554036 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:10.554010    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55e11ef7-bf87-4002-93e8-46af5da4c8f0" containerName="payload-pre-processing"
Oct 30 19:48:10.554036 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:10.554019    5907 cpu_manager.go:414] "containerMap: removing container" podUID="0fbf8d00-57a5-49ba-855a-4accc2f1bb05" containerName="istio-proxy"
Oct 30 19:48:10.554036 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:10.554023    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0fbf8d00-57a5-49ba-855a-4accc2f1bb05" containerName="istio-proxy"
Oct 30 19:48:10.554201 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:10.554127    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="fb5c110f-658c-46af-93b1-2122cc7b37b0" containerName="revoke-keys"
Oct 30 19:48:10.554201 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:10.554140    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="55e11ef7-bf87-4002-93e8-46af5da4c8f0" containerName="payload-pre-processing"
Oct 30 19:48:10.554201 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:10.554145    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="0fbf8d00-57a5-49ba-855a-4accc2f1bb05" containerName="istio-proxy"
Oct 30 19:48:10.559701 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:10.559669    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w5-3c89af-csp6c"
Oct 30 19:48:10.577196 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:10.577171    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:48:10.592905 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:10.590263    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:48:10.594114 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:10.594086    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w5-3c89af-csp6c"]
Oct 30 19:48:10.625194 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:10.625096    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/21bba6a0-8dc2-46d7-9a2d-3b2be380544a-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w5-3c89af-csp6c\" (UID: \"21bba6a0-8dc2-46d7-9a2d-3b2be380544a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w5-3c89af-csp6c"
Oct 30 19:48:10.625194 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:10.625146    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgcdl\" (UniqueName: \"kubernetes.io/projected/21bba6a0-8dc2-46d7-9a2d-3b2be380544a-kube-api-access-tgcdl\") pod \"maas-api-revoke-keys-e2e-shared-a-w5-3c89af-csp6c\" (UID: \"21bba6a0-8dc2-46d7-9a2d-3b2be380544a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w5-3c89af-csp6c"
Oct 30 19:48:10.625435 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:10.625230    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/21bba6a0-8dc2-46d7-9a2d-3b2be380544a-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w5-3c89af-csp6c\" (UID: \"21bba6a0-8dc2-46d7-9a2d-3b2be380544a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w5-3c89af-csp6c"
Oct 30 19:48:10.727154 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:10.727111    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/21bba6a0-8dc2-46d7-9a2d-3b2be380544a-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w5-3c89af-csp6c\" (UID: \"21bba6a0-8dc2-46d7-9a2d-3b2be380544a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w5-3c89af-csp6c"
Oct 30 19:48:10.727341 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:10.727160    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgcdl\" (UniqueName: \"kubernetes.io/projected/21bba6a0-8dc2-46d7-9a2d-3b2be380544a-kube-api-access-tgcdl\") pod \"maas-api-revoke-keys-e2e-shared-a-w5-3c89af-csp6c\" (UID: \"21bba6a0-8dc2-46d7-9a2d-3b2be380544a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w5-3c89af-csp6c"
Oct 30 19:48:10.727341 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:10.727190    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/21bba6a0-8dc2-46d7-9a2d-3b2be380544a-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w5-3c89af-csp6c\" (UID: \"21bba6a0-8dc2-46d7-9a2d-3b2be380544a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w5-3c89af-csp6c"
Oct 30 19:48:10.727912 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:10.727890    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/21bba6a0-8dc2-46d7-9a2d-3b2be380544a-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w5-3c89af-csp6c\" (UID: \"21bba6a0-8dc2-46d7-9a2d-3b2be380544a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w5-3c89af-csp6c"
Oct 30 19:48:10.729836 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:10.729811    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/21bba6a0-8dc2-46d7-9a2d-3b2be380544a-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w5-3c89af-csp6c\" (UID: \"21bba6a0-8dc2-46d7-9a2d-3b2be380544a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w5-3c89af-csp6c"
Oct 30 19:48:10.750831 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:10.750792    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgcdl\" (UniqueName: \"kubernetes.io/projected/21bba6a0-8dc2-46d7-9a2d-3b2be380544a-kube-api-access-tgcdl\") pod \"maas-api-revoke-keys-e2e-shared-a-w5-3c89af-csp6c\" (UID: \"21bba6a0-8dc2-46d7-9a2d-3b2be380544a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w5-3c89af-csp6c"
Oct 30 19:48:10.875845 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:10.875746    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w5-3c89af-csp6c"
Oct 30 19:48:11.189249 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:11.189020    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w5-3c89af-csp6c"]
Oct 30 19:48:11.191198 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:48:11.191158    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21bba6a0_8dc2_46d7_9a2d_3b2be380544a.slice/crio-aa8171e877a1804181d69c893c225904b4bc45b79eae7b677f06537e39517169 WatchSource:0}: Error finding container aa8171e877a1804181d69c893c225904b4bc45b79eae7b677f06537e39517169: Status 404 returned error can't find the container with id aa8171e877a1804181d69c893c225904b4bc45b79eae7b677f06537e39517169
Oct 30 19:48:11.442137 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:11.442067    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w5-3c89af-csp6c" event={"ID":"21bba6a0-8dc2-46d7-9a2d-3b2be380544a","Type":"ContainerStarted","Data":"aa8171e877a1804181d69c893c225904b4bc45b79eae7b677f06537e39517169"}
Oct 30 19:48:12.450638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:12.450602    5907 generic.go:356] "Generic (PLEG): container finished" podID="21bba6a0-8dc2-46d7-9a2d-3b2be380544a" containerID="ef4ff1c73b95baa47f3277815baafe4a52f32ad97f5eeeee37499e1e40d61fb7" exitCode=0
Oct 30 19:48:12.451072 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:12.450680    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w5-3c89af-csp6c" event={"ID":"21bba6a0-8dc2-46d7-9a2d-3b2be380544a","Type":"ContainerDied","Data":"ef4ff1c73b95baa47f3277815baafe4a52f32ad97f5eeeee37499e1e40d61fb7"}
Oct 30 19:48:13.639960 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:13.639936    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w5-3c89af-csp6c"
Oct 30 19:48:13.767320 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:13.767237    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/21bba6a0-8dc2-46d7-9a2d-3b2be380544a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/21bba6a0-8dc2-46d7-9a2d-3b2be380544a-maas-api-openshift-ca\") pod \"21bba6a0-8dc2-46d7-9a2d-3b2be380544a\" (UID: \"21bba6a0-8dc2-46d7-9a2d-3b2be380544a\") "
Oct 30 19:48:13.767481 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:13.767407    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/21bba6a0-8dc2-46d7-9a2d-3b2be380544a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/21bba6a0-8dc2-46d7-9a2d-3b2be380544a-maas-api-xks-ca\") pod \"21bba6a0-8dc2-46d7-9a2d-3b2be380544a\" (UID: \"21bba6a0-8dc2-46d7-9a2d-3b2be380544a\") "
Oct 30 19:48:13.767481 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:13.767457    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/21bba6a0-8dc2-46d7-9a2d-3b2be380544a-kube-api-access-tgcdl\" (UniqueName: \"kubernetes.io/projected/21bba6a0-8dc2-46d7-9a2d-3b2be380544a-kube-api-access-tgcdl\") pod \"21bba6a0-8dc2-46d7-9a2d-3b2be380544a\" (UID: \"21bba6a0-8dc2-46d7-9a2d-3b2be380544a\") "
Oct 30 19:48:13.767604 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:13.767579    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21bba6a0-8dc2-46d7-9a2d-3b2be380544a-maas-api-openshift-ca" pod "21bba6a0-8dc2-46d7-9a2d-3b2be380544a" (UID: "21bba6a0-8dc2-46d7-9a2d-3b2be380544a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:48:13.767844 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:13.767821    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/21bba6a0-8dc2-46d7-9a2d-3b2be380544a-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:13.769927 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:13.769897    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21bba6a0-8dc2-46d7-9a2d-3b2be380544a-kube-api-access-tgcdl" pod "21bba6a0-8dc2-46d7-9a2d-3b2be380544a" (UID: "21bba6a0-8dc2-46d7-9a2d-3b2be380544a"). InnerVolumeSpecName "kube-api-access-tgcdl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:13.770055 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:13.769923    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21bba6a0-8dc2-46d7-9a2d-3b2be380544a-maas-api-xks-ca" pod "21bba6a0-8dc2-46d7-9a2d-3b2be380544a" (UID: "21bba6a0-8dc2-46d7-9a2d-3b2be380544a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:48:13.869403 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:13.869363    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgcdl\" (UniqueName: \"kubernetes.io/projected/21bba6a0-8dc2-46d7-9a2d-3b2be380544a-kube-api-access-tgcdl\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:13.869403 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:13.869395    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/21bba6a0-8dc2-46d7-9a2d-3b2be380544a-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:14.466656 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:14.466617    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w5-3c89af-csp6c" event={"ID":"21bba6a0-8dc2-46d7-9a2d-3b2be380544a","Type":"ContainerDied","Data":"aa8171e877a1804181d69c893c225904b4bc45b79eae7b677f06537e39517169"}
Oct 30 19:48:14.466656 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:14.466652    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w5-3c89af-csp6c"
Oct 30 19:48:14.466894 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:14.466674    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aa8171e877a1804181d69c893c225904b4bc45b79eae7b677f06537e39517169"
Oct 30 19:48:15.930833 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:15.930801    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"]
Oct 30 19:48:15.931310 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:15.931045    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk" podUID="1b70d17b-10c5-4886-a033-caef763f1406" containerName="istio-proxy" containerID="cri-o://e9b111838bea8cfdc5ca9c46a1b78fe32a94c8e2c19c3a4cec0f2ead12f18ff4" gracePeriod=30
Oct 30 19:48:17.501144 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:48:17.501106    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:48:19.035509 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:19.035478    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6a544e22-rf86x"]
Oct 30 19:48:19.048041 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:19.048016    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6a544e22-rf86x"]
Oct 30 19:48:20.290066 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:20.287282    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2aa211ac-280d-4294-8f9d-807318fafceb" path="/var/lib/kubelet/pods/2aa211ac-280d-4294-8f9d-807318fafceb/volumes"
Oct 30 19:48:20.980518 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:20.980487    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w5-3c89af-f9bd48d78-wvb77"]
Oct 30 19:48:21.009044 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.009019    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3c89af-6d87768f7f-qxrpq"]
Oct 30 19:48:21.254377 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.254350    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:48:21.384624 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.384589    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b70d17b-10c5-4886-a033-caef763f1406-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b70d17b-10c5-4886-a033-caef763f1406-istio-data\") pod \"1b70d17b-10c5-4886-a033-caef763f1406\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") "
Oct 30 19:48:21.385119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.384659    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b70d17b-10c5-4886-a033-caef763f1406-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b70d17b-10c5-4886-a033-caef763f1406-workload-certs\") pod \"1b70d17b-10c5-4886-a033-caef763f1406\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") "
Oct 30 19:48:21.385119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.384697    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b70d17b-10c5-4886-a033-caef763f1406-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b70d17b-10c5-4886-a033-caef763f1406-workload-socket\") pod \"1b70d17b-10c5-4886-a033-caef763f1406\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") "
Oct 30 19:48:21.385119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.384805    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b70d17b-10c5-4886-a033-caef763f1406-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b70d17b-10c5-4886-a033-caef763f1406-istio-envoy\") pod \"1b70d17b-10c5-4886-a033-caef763f1406\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") "
Oct 30 19:48:21.385119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.384850    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1b70d17b-10c5-4886-a033-caef763f1406-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b70d17b-10c5-4886-a033-caef763f1406-istio-podinfo\") pod \"1b70d17b-10c5-4886-a033-caef763f1406\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") "
Oct 30 19:48:21.385119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.384959    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b70d17b-10c5-4886-a033-caef763f1406-istio-data" pod "1b70d17b-10c5-4886-a033-caef763f1406" (UID: "1b70d17b-10c5-4886-a033-caef763f1406"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:21.385119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.385013    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b70d17b-10c5-4886-a033-caef763f1406-workload-certs" pod "1b70d17b-10c5-4886-a033-caef763f1406" (UID: "1b70d17b-10c5-4886-a033-caef763f1406"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:21.385119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.385015    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1b70d17b-10c5-4886-a033-caef763f1406-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b70d17b-10c5-4886-a033-caef763f1406-istiod-ca-cert\") pod \"1b70d17b-10c5-4886-a033-caef763f1406\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") "
Oct 30 19:48:21.385119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.385099    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b70d17b-10c5-4886-a033-caef763f1406-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b70d17b-10c5-4886-a033-caef763f1406-credential-socket\") pod \"1b70d17b-10c5-4886-a033-caef763f1406\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") "
Oct 30 19:48:21.385119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.385112    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b70d17b-10c5-4886-a033-caef763f1406-workload-socket" pod "1b70d17b-10c5-4886-a033-caef763f1406" (UID: "1b70d17b-10c5-4886-a033-caef763f1406"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:21.385638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.385212    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1b70d17b-10c5-4886-a033-caef763f1406-kube-api-access-ttrln\" (UniqueName: \"kubernetes.io/projected/1b70d17b-10c5-4886-a033-caef763f1406-kube-api-access-ttrln\") pod \"1b70d17b-10c5-4886-a033-caef763f1406\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") "
Oct 30 19:48:21.385638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.385253    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1b70d17b-10c5-4886-a033-caef763f1406-istio-token\" (UniqueName: \"kubernetes.io/projected/1b70d17b-10c5-4886-a033-caef763f1406-istio-token\") pod \"1b70d17b-10c5-4886-a033-caef763f1406\" (UID: \"1b70d17b-10c5-4886-a033-caef763f1406\") "
Oct 30 19:48:21.385638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.385273    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b70d17b-10c5-4886-a033-caef763f1406-istiod-ca-cert" pod "1b70d17b-10c5-4886-a033-caef763f1406" (UID: "1b70d17b-10c5-4886-a033-caef763f1406"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:48:21.385638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.385369    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b70d17b-10c5-4886-a033-caef763f1406-credential-socket" pod "1b70d17b-10c5-4886-a033-caef763f1406" (UID: "1b70d17b-10c5-4886-a033-caef763f1406"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:21.385954 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.385806    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b70d17b-10c5-4886-a033-caef763f1406-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:21.385954 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.385830    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b70d17b-10c5-4886-a033-caef763f1406-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:21.385954 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.385882    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b70d17b-10c5-4886-a033-caef763f1406-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:21.385954 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.385913    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b70d17b-10c5-4886-a033-caef763f1406-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:21.386138 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.386008    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b70d17b-10c5-4886-a033-caef763f1406-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:21.387419 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.387397    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1b70d17b-10c5-4886-a033-caef763f1406-istio-podinfo" pod "1b70d17b-10c5-4886-a033-caef763f1406" (UID: "1b70d17b-10c5-4886-a033-caef763f1406"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:48:21.387514 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.387465    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b70d17b-10c5-4886-a033-caef763f1406-istio-token" pod "1b70d17b-10c5-4886-a033-caef763f1406" (UID: "1b70d17b-10c5-4886-a033-caef763f1406"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:21.387683 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.387667    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b70d17b-10c5-4886-a033-caef763f1406-istio-envoy" pod "1b70d17b-10c5-4886-a033-caef763f1406" (UID: "1b70d17b-10c5-4886-a033-caef763f1406"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:21.387822 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.387799    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b70d17b-10c5-4886-a033-caef763f1406-kube-api-access-ttrln" pod "1b70d17b-10c5-4886-a033-caef763f1406" (UID: "1b70d17b-10c5-4886-a033-caef763f1406"). InnerVolumeSpecName "kube-api-access-ttrln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:21.487154 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.487079    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ttrln\" (UniqueName: \"kubernetes.io/projected/1b70d17b-10c5-4886-a033-caef763f1406-kube-api-access-ttrln\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:21.487154 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.487104    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b70d17b-10c5-4886-a033-caef763f1406-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:21.487154 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.487113    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b70d17b-10c5-4886-a033-caef763f1406-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:21.487154 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.487122    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b70d17b-10c5-4886-a033-caef763f1406-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:21.537833 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.537798    5907 generic.go:356] "Generic (PLEG): container finished" podID="1b70d17b-10c5-4886-a033-caef763f1406" containerID="e9b111838bea8cfdc5ca9c46a1b78fe32a94c8e2c19c3a4cec0f2ead12f18ff4" exitCode=0
Oct 30 19:48:21.538052 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.537851    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"
Oct 30 19:48:21.538052 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.537915    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk" event={"ID":"1b70d17b-10c5-4886-a033-caef763f1406","Type":"ContainerDied","Data":"e9b111838bea8cfdc5ca9c46a1b78fe32a94c8e2c19c3a4cec0f2ead12f18ff4"}
Oct 30 19:48:21.538052 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.537982    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk" event={"ID":"1b70d17b-10c5-4886-a033-caef763f1406","Type":"ContainerDied","Data":"f68109273bf540cf4e07952460b4b3e37a6f18b0fccca5ca72492a0b5b16bea7"}
Oct 30 19:48:21.538052 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.538007    5907 scope.go:122] "RemoveContainer" containerID="e9b111838bea8cfdc5ca9c46a1b78fe32a94c8e2c19c3a4cec0f2ead12f18ff4"
Oct 30 19:48:21.592135 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.580120    5907 scope.go:122] "RemoveContainer" containerID="c34293f861a664a7de82259386c8ba601b042478b12ea37bf1d692111d7a5349"
Oct 30 19:48:21.611389 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.611363    5907 scope.go:122] "RemoveContainer" containerID="e9b111838bea8cfdc5ca9c46a1b78fe32a94c8e2c19c3a4cec0f2ead12f18ff4"
Oct 30 19:48:21.612432 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:48:21.612406    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9b111838bea8cfdc5ca9c46a1b78fe32a94c8e2c19c3a4cec0f2ead12f18ff4\": container with ID starting with e9b111838bea8cfdc5ca9c46a1b78fe32a94c8e2c19c3a4cec0f2ead12f18ff4 not found: ID does not exist" containerID="e9b111838bea8cfdc5ca9c46a1b78fe32a94c8e2c19c3a4cec0f2ead12f18ff4"
Oct 30 19:48:21.612576 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.612445    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9b111838bea8cfdc5ca9c46a1b78fe32a94c8e2c19c3a4cec0f2ead12f18ff4"} err="failed to get container status \"e9b111838bea8cfdc5ca9c46a1b78fe32a94c8e2c19c3a4cec0f2ead12f18ff4\": rpc error: code = NotFound desc = could not find container \"e9b111838bea8cfdc5ca9c46a1b78fe32a94c8e2c19c3a4cec0f2ead12f18ff4\": container with ID starting with e9b111838bea8cfdc5ca9c46a1b78fe32a94c8e2c19c3a4cec0f2ead12f18ff4 not found: ID does not exist"
Oct 30 19:48:21.612576 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.612471    5907 scope.go:122] "RemoveContainer" containerID="c34293f861a664a7de82259386c8ba601b042478b12ea37bf1d692111d7a5349"
Oct 30 19:48:21.612894 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:48:21.612834    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c34293f861a664a7de82259386c8ba601b042478b12ea37bf1d692111d7a5349\": container with ID starting with c34293f861a664a7de82259386c8ba601b042478b12ea37bf1d692111d7a5349 not found: ID does not exist" containerID="c34293f861a664a7de82259386c8ba601b042478b12ea37bf1d692111d7a5349"
Oct 30 19:48:21.612995 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.612912    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c34293f861a664a7de82259386c8ba601b042478b12ea37bf1d692111d7a5349"} err="failed to get container status \"c34293f861a664a7de82259386c8ba601b042478b12ea37bf1d692111d7a5349\": rpc error: code = NotFound desc = could not find container \"c34293f861a664a7de82259386c8ba601b042478b12ea37bf1d692111d7a5349\": container with ID starting with c34293f861a664a7de82259386c8ba601b042478b12ea37bf1d692111d7a5349 not found: ID does not exist"
Oct 30 19:48:21.645815 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.645338    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"]
Oct 30 19:48:21.648812 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.648784    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-d797d2-openshift-default-6dbbffdb8f-vcstk"]
Oct 30 19:48:21.965163 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:21.965130    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:48:22.287417 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:22.287333    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1b70d17b-10c5-4886-a033-caef763f1406" path="/var/lib/kubelet/pods/1b70d17b-10c5-4886-a033-caef763f1406/volumes"
Oct 30 19:48:25.651066 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:25.651028    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w5-3c89af-csp6c"]
Oct 30 19:48:25.664081 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:25.664050    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w5-3c89af-csp6c"]
Oct 30 19:48:25.981765 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:25.981667    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3c89af-f9bd48d78-wvb77" podUID="70810131-ad32-4a06-b69c-6142147dc2d9" containerName="payload-processing" containerID="cri-o://8c25860adf6a56b7b65fcc5d17fe92151a534d689e6adac22c2a72417d2c4e4b" gracePeriod=25
Oct 30 19:48:26.010266 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:26.010232    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3c89af-6d87768f7f-qxrpq" podUID="486e38b1-f36d-48b9-ad1f-5ac88f641c49" containerName="payload-pre-processing" containerID="cri-o://8db1de37c378554f7b340a84d1da1e820932e5e62af9e21e076d99b9fa1778f9" gracePeriod=25
Oct 30 19:48:26.276063 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:26.276031    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3c89af-f9bd48d78-wvb77"
Oct 30 19:48:26.286969 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:26.286940    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="21bba6a0-8dc2-46d7-9a2d-3b2be380544a" path="/var/lib/kubelet/pods/21bba6a0-8dc2-46d7-9a2d-3b2be380544a/volumes"
Oct 30 19:48:26.314179 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:26.314154    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3c89af-6d87768f7f-qxrpq"
Oct 30 19:48:26.355057 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:26.355023    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/486e38b1-f36d-48b9-ad1f-5ac88f641c49-kube-api-access-wq4vx\" (UniqueName: \"kubernetes.io/projected/486e38b1-f36d-48b9-ad1f-5ac88f641c49-kube-api-access-wq4vx\") pod \"486e38b1-f36d-48b9-ad1f-5ac88f641c49\" (UID: \"486e38b1-f36d-48b9-ad1f-5ac88f641c49\") "
Oct 30 19:48:26.355242 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:26.355094    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/70810131-ad32-4a06-b69c-6142147dc2d9-kube-api-access-shnxh\" (UniqueName: \"kubernetes.io/projected/70810131-ad32-4a06-b69c-6142147dc2d9-kube-api-access-shnxh\") pod \"70810131-ad32-4a06-b69c-6142147dc2d9\" (UID: \"70810131-ad32-4a06-b69c-6142147dc2d9\") "
Oct 30 19:48:26.355242 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:26.355234    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/486e38b1-f36d-48b9-ad1f-5ac88f641c49-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/486e38b1-f36d-48b9-ad1f-5ac88f641c49-plugins-config-volume\") pod \"486e38b1-f36d-48b9-ad1f-5ac88f641c49\" (UID: \"486e38b1-f36d-48b9-ad1f-5ac88f641c49\") "
Oct 30 19:48:26.355351 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:26.355265    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/70810131-ad32-4a06-b69c-6142147dc2d9-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70810131-ad32-4a06-b69c-6142147dc2d9-plugins-config-volume\") pod \"70810131-ad32-4a06-b69c-6142147dc2d9\" (UID: \"70810131-ad32-4a06-b69c-6142147dc2d9\") "
Oct 30 19:48:26.355623 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:26.355595    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/486e38b1-f36d-48b9-ad1f-5ac88f641c49-plugins-config-volume" pod "486e38b1-f36d-48b9-ad1f-5ac88f641c49" (UID: "486e38b1-f36d-48b9-ad1f-5ac88f641c49"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:48:26.355738 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:26.355687    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70810131-ad32-4a06-b69c-6142147dc2d9-plugins-config-volume" pod "70810131-ad32-4a06-b69c-6142147dc2d9" (UID: "70810131-ad32-4a06-b69c-6142147dc2d9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:48:26.356160 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:26.356137    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/486e38b1-f36d-48b9-ad1f-5ac88f641c49-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:26.356265 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:26.356167    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70810131-ad32-4a06-b69c-6142147dc2d9-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:26.357736 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:26.357711    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70810131-ad32-4a06-b69c-6142147dc2d9-kube-api-access-shnxh" pod "70810131-ad32-4a06-b69c-6142147dc2d9" (UID: "70810131-ad32-4a06-b69c-6142147dc2d9"). InnerVolumeSpecName "kube-api-access-shnxh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:26.357943 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:26.357790    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/486e38b1-f36d-48b9-ad1f-5ac88f641c49-kube-api-access-wq4vx" pod "486e38b1-f36d-48b9-ad1f-5ac88f641c49" (UID: "486e38b1-f36d-48b9-ad1f-5ac88f641c49"). InnerVolumeSpecName "kube-api-access-wq4vx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:26.457037 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:26.457000    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wq4vx\" (UniqueName: \"kubernetes.io/projected/486e38b1-f36d-48b9-ad1f-5ac88f641c49-kube-api-access-wq4vx\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:26.457037 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:26.457029    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shnxh\" (UniqueName: \"kubernetes.io/projected/70810131-ad32-4a06-b69c-6142147dc2d9-kube-api-access-shnxh\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:26.585275 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:26.585196    5907 generic.go:356] "Generic (PLEG): container finished" podID="486e38b1-f36d-48b9-ad1f-5ac88f641c49" containerID="8db1de37c378554f7b340a84d1da1e820932e5e62af9e21e076d99b9fa1778f9" exitCode=0
Oct 30 19:48:26.585275 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:26.585235    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3c89af-6d87768f7f-qxrpq"
Oct 30 19:48:26.585493 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:26.585280    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3c89af-6d87768f7f-qxrpq" event={"ID":"486e38b1-f36d-48b9-ad1f-5ac88f641c49","Type":"ContainerDied","Data":"8db1de37c378554f7b340a84d1da1e820932e5e62af9e21e076d99b9fa1778f9"}
Oct 30 19:48:26.585493 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:26.585341    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3c89af-6d87768f7f-qxrpq" event={"ID":"486e38b1-f36d-48b9-ad1f-5ac88f641c49","Type":"ContainerDied","Data":"2844e32e4c1136a036ed232ea47d075346a619e634ae10a045fac3cdb1fda49c"}
Oct 30 19:48:26.585493 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:26.585370    5907 scope.go:122] "RemoveContainer" containerID="8db1de37c378554f7b340a84d1da1e820932e5e62af9e21e076d99b9fa1778f9"
Oct 30 19:48:26.587982 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:26.587957    5907 generic.go:356] "Generic (PLEG): container finished" podID="70810131-ad32-4a06-b69c-6142147dc2d9" containerID="8c25860adf6a56b7b65fcc5d17fe92151a534d689e6adac22c2a72417d2c4e4b" exitCode=0
Oct 30 19:48:26.588122 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:26.588008    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3c89af-f9bd48d78-wvb77"
Oct 30 19:48:26.588266 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:26.588011    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3c89af-f9bd48d78-wvb77" event={"ID":"70810131-ad32-4a06-b69c-6142147dc2d9","Type":"ContainerDied","Data":"8c25860adf6a56b7b65fcc5d17fe92151a534d689e6adac22c2a72417d2c4e4b"}
Oct 30 19:48:26.588340 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:26.588300    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-3c89af-f9bd48d78-wvb77" event={"ID":"70810131-ad32-4a06-b69c-6142147dc2d9","Type":"ContainerDied","Data":"6620365f8d751405f98931bd9ef91a45500baaaf1fdf460ab70cdaeb8f452305"}
Oct 30 19:48:26.597978 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:26.597907    5907 scope.go:122] "RemoveContainer" containerID="8db1de37c378554f7b340a84d1da1e820932e5e62af9e21e076d99b9fa1778f9"
Oct 30 19:48:26.598418 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:48:26.598389    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8db1de37c378554f7b340a84d1da1e820932e5e62af9e21e076d99b9fa1778f9\": container with ID starting with 8db1de37c378554f7b340a84d1da1e820932e5e62af9e21e076d99b9fa1778f9 not found: ID does not exist" containerID="8db1de37c378554f7b340a84d1da1e820932e5e62af9e21e076d99b9fa1778f9"
Oct 30 19:48:26.598561 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:26.598450    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8db1de37c378554f7b340a84d1da1e820932e5e62af9e21e076d99b9fa1778f9"} err="failed to get container status \"8db1de37c378554f7b340a84d1da1e820932e5e62af9e21e076d99b9fa1778f9\": rpc error: code = NotFound desc = could not find container \"8db1de37c378554f7b340a84d1da1e820932e5e62af9e21e076d99b9fa1778f9\": container with ID starting with 8db1de37c378554f7b340a84d1da1e820932e5e62af9e21e076d99b9fa1778f9 not found: ID does not exist"
Oct 30 19:48:26.598561 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:26.598474    5907 scope.go:122] "RemoveContainer" containerID="8c25860adf6a56b7b65fcc5d17fe92151a534d689e6adac22c2a72417d2c4e4b"
Oct 30 19:48:26.609122 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:26.609102    5907 scope.go:122] "RemoveContainer" containerID="8c25860adf6a56b7b65fcc5d17fe92151a534d689e6adac22c2a72417d2c4e4b"
Oct 30 19:48:26.609450 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:48:26.609429    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c25860adf6a56b7b65fcc5d17fe92151a534d689e6adac22c2a72417d2c4e4b\": container with ID starting with 8c25860adf6a56b7b65fcc5d17fe92151a534d689e6adac22c2a72417d2c4e4b not found: ID does not exist" containerID="8c25860adf6a56b7b65fcc5d17fe92151a534d689e6adac22c2a72417d2c4e4b"
Oct 30 19:48:26.609532 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:26.609460    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c25860adf6a56b7b65fcc5d17fe92151a534d689e6adac22c2a72417d2c4e4b"} err="failed to get container status \"8c25860adf6a56b7b65fcc5d17fe92151a534d689e6adac22c2a72417d2c4e4b\": rpc error: code = NotFound desc = could not find container \"8c25860adf6a56b7b65fcc5d17fe92151a534d689e6adac22c2a72417d2c4e4b\": container with ID starting with 8c25860adf6a56b7b65fcc5d17fe92151a534d689e6adac22c2a72417d2c4e4b not found: ID does not exist"
Oct 30 19:48:26.642531 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:26.642503    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3c89af-6d87768f7f-qxrpq"]
Oct 30 19:48:26.655571 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:26.655545    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-3c89af-6d87768f7f-qxrpq"]
Oct 30 19:48:26.692179 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:26.692146    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w5-3c89af-f9bd48d78-wvb77"]
Oct 30 19:48:26.694634 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:26.694609    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w5-3c89af-f9bd48d78-wvb77"]
Oct 30 19:48:28.287642 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:28.287606    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="486e38b1-f36d-48b9-ad1f-5ac88f641c49" path="/var/lib/kubelet/pods/486e38b1-f36d-48b9-ad1f-5ac88f641c49/volumes"
Oct 30 19:48:28.288048 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:28.288000    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="70810131-ad32-4a06-b69c-6142147dc2d9" path="/var/lib/kubelet/pods/70810131-ad32-4a06-b69c-6142147dc2d9/volumes"
Oct 30 19:48:28.427818 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:28.427775    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:48:32.390300 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:32.390268    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"]
Oct 30 19:48:32.390737 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:32.390521    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp" podUID="6662d5b3-a2a2-4187-ae74-afaeaa1b82e4" containerName="istio-proxy" containerID="cri-o://269c03112689e83620338c8c4dcba3c0e6bdad7380c401ad8aa58735fb6b3299" gracePeriod=30
Oct 30 19:48:32.961837 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:32.961802    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w5-e9de60-fwccr"]
Oct 30 19:48:32.962463 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:32.962438    5907 cpu_manager.go:414] "containerMap: removing container" podUID="1b70d17b-10c5-4886-a033-caef763f1406" containerName="istio-proxy"
Oct 30 19:48:32.962463 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:32.962458    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1b70d17b-10c5-4886-a033-caef763f1406" containerName="istio-proxy"
Oct 30 19:48:32.962463 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:32.962471    5907 cpu_manager.go:414] "containerMap: removing container" podUID="1b70d17b-10c5-4886-a033-caef763f1406" containerName="istio-proxy"
Oct 30 19:48:32.962753 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:32.962477    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1b70d17b-10c5-4886-a033-caef763f1406" containerName="istio-proxy"
Oct 30 19:48:32.962753 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:32.962492    5907 cpu_manager.go:414] "containerMap: removing container" podUID="70810131-ad32-4a06-b69c-6142147dc2d9" containerName="payload-processing"
Oct 30 19:48:32.962753 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:32.962497    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="70810131-ad32-4a06-b69c-6142147dc2d9" containerName="payload-processing"
Oct 30 19:48:32.962753 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:32.962509    5907 cpu_manager.go:414] "containerMap: removing container" podUID="21bba6a0-8dc2-46d7-9a2d-3b2be380544a" containerName="revoke-keys"
Oct 30 19:48:32.962753 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:32.962514    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="21bba6a0-8dc2-46d7-9a2d-3b2be380544a" containerName="revoke-keys"
Oct 30 19:48:32.962753 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:32.962525    5907 cpu_manager.go:414] "containerMap: removing container" podUID="486e38b1-f36d-48b9-ad1f-5ac88f641c49" containerName="payload-pre-processing"
Oct 30 19:48:32.962753 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:32.962532    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="486e38b1-f36d-48b9-ad1f-5ac88f641c49" containerName="payload-pre-processing"
Oct 30 19:48:32.962753 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:32.962612    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="1b70d17b-10c5-4886-a033-caef763f1406" containerName="istio-proxy"
Oct 30 19:48:32.962753 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:32.962620    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="1b70d17b-10c5-4886-a033-caef763f1406" containerName="istio-proxy"
Oct 30 19:48:32.962753 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:32.962630    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="70810131-ad32-4a06-b69c-6142147dc2d9" containerName="payload-processing"
Oct 30 19:48:32.962753 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:32.962636    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="21bba6a0-8dc2-46d7-9a2d-3b2be380544a" containerName="revoke-keys"
Oct 30 19:48:32.962753 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:32.962644    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="486e38b1-f36d-48b9-ad1f-5ac88f641c49" containerName="payload-pre-processing"
Oct 30 19:48:32.972480 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:32.972458    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w5-e9de60-fwccr"
Oct 30 19:48:32.986065 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:32.986039    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w5-e9de60-fwccr"]
Oct 30 19:48:33.036253 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:33.036219    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/90aa7fc8-86b9-48fe-a573-78780f3503cd-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w5-e9de60-fwccr\" (UID: \"90aa7fc8-86b9-48fe-a573-78780f3503cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w5-e9de60-fwccr"
Oct 30 19:48:33.036450 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:33.036336    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7nxt\" (UniqueName: \"kubernetes.io/projected/90aa7fc8-86b9-48fe-a573-78780f3503cd-kube-api-access-w7nxt\") pod \"maas-api-revoke-keys-e2e-shared-b-w5-e9de60-fwccr\" (UID: \"90aa7fc8-86b9-48fe-a573-78780f3503cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w5-e9de60-fwccr"
Oct 30 19:48:33.036450 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:33.036376    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/90aa7fc8-86b9-48fe-a573-78780f3503cd-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w5-e9de60-fwccr\" (UID: \"90aa7fc8-86b9-48fe-a573-78780f3503cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w5-e9de60-fwccr"
Oct 30 19:48:33.067976 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:33.067948    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:48:33.068117 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:33.068037    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:48:33.137473 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:33.137436    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/90aa7fc8-86b9-48fe-a573-78780f3503cd-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w5-e9de60-fwccr\" (UID: \"90aa7fc8-86b9-48fe-a573-78780f3503cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w5-e9de60-fwccr"
Oct 30 19:48:33.137643 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:33.137517    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7nxt\" (UniqueName: \"kubernetes.io/projected/90aa7fc8-86b9-48fe-a573-78780f3503cd-kube-api-access-w7nxt\") pod \"maas-api-revoke-keys-e2e-shared-b-w5-e9de60-fwccr\" (UID: \"90aa7fc8-86b9-48fe-a573-78780f3503cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w5-e9de60-fwccr"
Oct 30 19:48:33.137643 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:33.137562    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/90aa7fc8-86b9-48fe-a573-78780f3503cd-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w5-e9de60-fwccr\" (UID: \"90aa7fc8-86b9-48fe-a573-78780f3503cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w5-e9de60-fwccr"
Oct 30 19:48:33.138445 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:33.138418    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/90aa7fc8-86b9-48fe-a573-78780f3503cd-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w5-e9de60-fwccr\" (UID: \"90aa7fc8-86b9-48fe-a573-78780f3503cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w5-e9de60-fwccr"
Oct 30 19:48:33.139747 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:33.139725    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/90aa7fc8-86b9-48fe-a573-78780f3503cd-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w5-e9de60-fwccr\" (UID: \"90aa7fc8-86b9-48fe-a573-78780f3503cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w5-e9de60-fwccr"
Oct 30 19:48:33.159187 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:33.159163    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7nxt\" (UniqueName: \"kubernetes.io/projected/90aa7fc8-86b9-48fe-a573-78780f3503cd-kube-api-access-w7nxt\") pod \"maas-api-revoke-keys-e2e-shared-b-w5-e9de60-fwccr\" (UID: \"90aa7fc8-86b9-48fe-a573-78780f3503cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w5-e9de60-fwccr"
Oct 30 19:48:33.288088 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:33.288010    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w5-e9de60-fwccr"
Oct 30 19:48:33.595279 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:33.595255    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w5-e9de60-fwccr"]
Oct 30 19:48:33.597191 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:48:33.597163    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90aa7fc8_86b9_48fe_a573_78780f3503cd.slice/crio-1d2d477d2f6b6aeac70085e95fc7c6ffd2e8afc874550ba79608630bc4b98198 WatchSource:0}: Error finding container 1d2d477d2f6b6aeac70085e95fc7c6ffd2e8afc874550ba79608630bc4b98198: Status 404 returned error can't find the container with id 1d2d477d2f6b6aeac70085e95fc7c6ffd2e8afc874550ba79608630bc4b98198
Oct 30 19:48:33.642829 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:33.642797    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w5-e9de60-fwccr" event={"ID":"90aa7fc8-86b9-48fe-a573-78780f3503cd","Type":"ContainerStarted","Data":"1d2d477d2f6b6aeac70085e95fc7c6ffd2e8afc874550ba79608630bc4b98198"}
Oct 30 19:48:33.971371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:33.971344    5907 scope.go:122] "RemoveContainer" containerID="0ec5e3e105c41201d3d8979a08a3631d8fedf4b6e741d0df86b238c049f44a11"
Oct 30 19:48:33.981051 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:33.981031    5907 scope.go:122] "RemoveContainer" containerID="e70e17cf0457faea617aa7028cd2877bcf5f1e85f72bf87fa68fb5f3279b91ef"
Oct 30 19:48:34.651015 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:34.650982    5907 generic.go:356] "Generic (PLEG): container finished" podID="90aa7fc8-86b9-48fe-a573-78780f3503cd" containerID="5b5244c8150d9d0ad6d27e7d36057ab054193986d0cac7633e0ad426a3ee7ac6" exitCode=0
Oct 30 19:48:34.651435 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:34.651061    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w5-e9de60-fwccr" event={"ID":"90aa7fc8-86b9-48fe-a573-78780f3503cd","Type":"ContainerDied","Data":"5b5244c8150d9d0ad6d27e7d36057ab054193986d0cac7633e0ad426a3ee7ac6"}
Oct 30 19:48:35.261650 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:35.261607    5907 patch_prober.go:29] interesting pod/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:48:35.261819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:35.261669    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp" podUID="6662d5b3-a2a2-4187-ae74-afaeaa1b82e4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:48:35.798514 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:35.798488    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w5-e9de60-fwccr"
Oct 30 19:48:35.865422 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:35.865385    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/90aa7fc8-86b9-48fe-a573-78780f3503cd-kube-api-access-w7nxt\" (UniqueName: \"kubernetes.io/projected/90aa7fc8-86b9-48fe-a573-78780f3503cd-kube-api-access-w7nxt\") pod \"90aa7fc8-86b9-48fe-a573-78780f3503cd\" (UID: \"90aa7fc8-86b9-48fe-a573-78780f3503cd\") "
Oct 30 19:48:35.865612 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:35.865438    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/90aa7fc8-86b9-48fe-a573-78780f3503cd-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/90aa7fc8-86b9-48fe-a573-78780f3503cd-maas-api-openshift-ca\") pod \"90aa7fc8-86b9-48fe-a573-78780f3503cd\" (UID: \"90aa7fc8-86b9-48fe-a573-78780f3503cd\") "
Oct 30 19:48:35.865612 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:35.865487    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/90aa7fc8-86b9-48fe-a573-78780f3503cd-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/90aa7fc8-86b9-48fe-a573-78780f3503cd-maas-api-xks-ca\") pod \"90aa7fc8-86b9-48fe-a573-78780f3503cd\" (UID: \"90aa7fc8-86b9-48fe-a573-78780f3503cd\") "
Oct 30 19:48:35.865838 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:35.865812    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90aa7fc8-86b9-48fe-a573-78780f3503cd-maas-api-openshift-ca" pod "90aa7fc8-86b9-48fe-a573-78780f3503cd" (UID: "90aa7fc8-86b9-48fe-a573-78780f3503cd"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:48:35.867848 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:35.867825    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90aa7fc8-86b9-48fe-a573-78780f3503cd-kube-api-access-w7nxt" pod "90aa7fc8-86b9-48fe-a573-78780f3503cd" (UID: "90aa7fc8-86b9-48fe-a573-78780f3503cd"). InnerVolumeSpecName "kube-api-access-w7nxt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:35.868120 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:35.868089    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90aa7fc8-86b9-48fe-a573-78780f3503cd-maas-api-xks-ca" pod "90aa7fc8-86b9-48fe-a573-78780f3503cd" (UID: "90aa7fc8-86b9-48fe-a573-78780f3503cd"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:48:35.966930 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:35.966814    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7nxt\" (UniqueName: \"kubernetes.io/projected/90aa7fc8-86b9-48fe-a573-78780f3503cd-kube-api-access-w7nxt\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:35.966930 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:35.966845    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/90aa7fc8-86b9-48fe-a573-78780f3503cd-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:35.966930 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:35.966888    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/90aa7fc8-86b9-48fe-a573-78780f3503cd-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:36.665470 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:36.665442    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w5-e9de60-fwccr"
Oct 30 19:48:36.665684 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:36.665441    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w5-e9de60-fwccr" event={"ID":"90aa7fc8-86b9-48fe-a573-78780f3503cd","Type":"ContainerDied","Data":"1d2d477d2f6b6aeac70085e95fc7c6ffd2e8afc874550ba79608630bc4b98198"}
Oct 30 19:48:36.665684 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:36.665574    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1d2d477d2f6b6aeac70085e95fc7c6ffd2e8afc874550ba79608630bc4b98198"
Oct 30 19:48:37.644152 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.644126    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:48:37.673984 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.673911    5907 generic.go:356] "Generic (PLEG): container finished" podID="6662d5b3-a2a2-4187-ae74-afaeaa1b82e4" containerID="269c03112689e83620338c8c4dcba3c0e6bdad7380c401ad8aa58735fb6b3299" exitCode=0
Oct 30 19:48:37.673984 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.673953    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"
Oct 30 19:48:37.674226 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.673990    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp" event={"ID":"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4","Type":"ContainerDied","Data":"269c03112689e83620338c8c4dcba3c0e6bdad7380c401ad8aa58735fb6b3299"}
Oct 30 19:48:37.674226 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.674038    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp" event={"ID":"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4","Type":"ContainerDied","Data":"3068626d2ec4b4a0eda5e6a5eed6fcd79933e5e4106905fd7c1ebc122ff51dca"}
Oct 30 19:48:37.674226 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.674061    5907 scope.go:122] "RemoveContainer" containerID="269c03112689e83620338c8c4dcba3c0e6bdad7380c401ad8aa58735fb6b3299"
Oct 30 19:48:37.684134 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.684113    5907 scope.go:122] "RemoveContainer" containerID="148ea3c66d89c893e746c8d7bda4d25f4ca0c29824aaa93c6389a86d90bca3be"
Oct 30 19:48:37.684670 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.684652    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-credential-socket\") pod \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") "
Oct 30 19:48:37.684738 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.684695    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-istio-envoy\") pod \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") "
Oct 30 19:48:37.684782 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.684737    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-istio-podinfo\") pod \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") "
Oct 30 19:48:37.684843 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.684781    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-workload-socket\") pod \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") "
Oct 30 19:48:37.684927 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.684843    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-kube-api-access-t8kvj\" (UniqueName: \"kubernetes.io/projected/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-kube-api-access-t8kvj\") pod \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") "
Oct 30 19:48:37.684993 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.684930    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-istio-data\") pod \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") "
Oct 30 19:48:37.684993 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.684955    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-istiod-ca-cert\") pod \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") "
Oct 30 19:48:37.685101 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.685044    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-istio-token\" (UniqueName: \"kubernetes.io/projected/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-istio-token\") pod \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") "
Oct 30 19:48:37.685101 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.685071    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-workload-certs\") pod \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\" (UID: \"6662d5b3-a2a2-4187-ae74-afaeaa1b82e4\") "
Oct 30 19:48:37.685216 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.685140    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-workload-socket" pod "6662d5b3-a2a2-4187-ae74-afaeaa1b82e4" (UID: "6662d5b3-a2a2-4187-ae74-afaeaa1b82e4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:37.685348 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.685236    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-credential-socket" pod "6662d5b3-a2a2-4187-ae74-afaeaa1b82e4" (UID: "6662d5b3-a2a2-4187-ae74-afaeaa1b82e4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:37.685435 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.685412    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-workload-certs" pod "6662d5b3-a2a2-4187-ae74-afaeaa1b82e4" (UID: "6662d5b3-a2a2-4187-ae74-afaeaa1b82e4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:37.685520 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.685467    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-istiod-ca-cert" pod "6662d5b3-a2a2-4187-ae74-afaeaa1b82e4" (UID: "6662d5b3-a2a2-4187-ae74-afaeaa1b82e4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:48:37.685729 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.685658    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-istio-data" pod "6662d5b3-a2a2-4187-ae74-afaeaa1b82e4" (UID: "6662d5b3-a2a2-4187-ae74-afaeaa1b82e4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:37.685786 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.685726    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:37.685786 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.685739    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:37.685786 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.685749    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:37.685786 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.685757    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:37.687818 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.687205    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-istio-podinfo" pod "6662d5b3-a2a2-4187-ae74-afaeaa1b82e4" (UID: "6662d5b3-a2a2-4187-ae74-afaeaa1b82e4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:48:37.687818 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.687748    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-kube-api-access-t8kvj" pod "6662d5b3-a2a2-4187-ae74-afaeaa1b82e4" (UID: "6662d5b3-a2a2-4187-ae74-afaeaa1b82e4"). InnerVolumeSpecName "kube-api-access-t8kvj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:37.687818 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.687774    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-istio-token" pod "6662d5b3-a2a2-4187-ae74-afaeaa1b82e4" (UID: "6662d5b3-a2a2-4187-ae74-afaeaa1b82e4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:37.687818 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.687814    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-istio-envoy" pod "6662d5b3-a2a2-4187-ae74-afaeaa1b82e4" (UID: "6662d5b3-a2a2-4187-ae74-afaeaa1b82e4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:37.714128 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.714098    5907 scope.go:122] "RemoveContainer" containerID="269c03112689e83620338c8c4dcba3c0e6bdad7380c401ad8aa58735fb6b3299"
Oct 30 19:48:37.714519 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:48:37.714499    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"269c03112689e83620338c8c4dcba3c0e6bdad7380c401ad8aa58735fb6b3299\": container with ID starting with 269c03112689e83620338c8c4dcba3c0e6bdad7380c401ad8aa58735fb6b3299 not found: ID does not exist" containerID="269c03112689e83620338c8c4dcba3c0e6bdad7380c401ad8aa58735fb6b3299"
Oct 30 19:48:37.714601 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.714529    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"269c03112689e83620338c8c4dcba3c0e6bdad7380c401ad8aa58735fb6b3299"} err="failed to get container status \"269c03112689e83620338c8c4dcba3c0e6bdad7380c401ad8aa58735fb6b3299\": rpc error: code = NotFound desc = could not find container \"269c03112689e83620338c8c4dcba3c0e6bdad7380c401ad8aa58735fb6b3299\": container with ID starting with 269c03112689e83620338c8c4dcba3c0e6bdad7380c401ad8aa58735fb6b3299 not found: ID does not exist"
Oct 30 19:48:37.714601 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.714548    5907 scope.go:122] "RemoveContainer" containerID="148ea3c66d89c893e746c8d7bda4d25f4ca0c29824aaa93c6389a86d90bca3be"
Oct 30 19:48:37.714912 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:48:37.714894    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"148ea3c66d89c893e746c8d7bda4d25f4ca0c29824aaa93c6389a86d90bca3be\": container with ID starting with 148ea3c66d89c893e746c8d7bda4d25f4ca0c29824aaa93c6389a86d90bca3be not found: ID does not exist" containerID="148ea3c66d89c893e746c8d7bda4d25f4ca0c29824aaa93c6389a86d90bca3be"
Oct 30 19:48:37.714992 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.714916    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"148ea3c66d89c893e746c8d7bda4d25f4ca0c29824aaa93c6389a86d90bca3be"} err="failed to get container status \"148ea3c66d89c893e746c8d7bda4d25f4ca0c29824aaa93c6389a86d90bca3be\": rpc error: code = NotFound desc = could not find container \"148ea3c66d89c893e746c8d7bda4d25f4ca0c29824aaa93c6389a86d90bca3be\": container with ID starting with 148ea3c66d89c893e746c8d7bda4d25f4ca0c29824aaa93c6389a86d90bca3be not found: ID does not exist"
Oct 30 19:48:37.786965 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.786932    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8kvj\" (UniqueName: \"kubernetes.io/projected/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-kube-api-access-t8kvj\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:37.786965 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.786960    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:37.786965 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.786970    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:37.787179 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.786978    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:37.787179 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:37.786986    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:38.034742 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:38.034708    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"]
Oct 30 19:48:38.057894 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:38.057835    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-3c89af-openshift-default-6dcd49dd5c-mlkgp"]
Oct 30 19:48:38.289000 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:38.288885    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6662d5b3-a2a2-4187-ae74-afaeaa1b82e4" path="/var/lib/kubelet/pods/6662d5b3-a2a2-4187-ae74-afaeaa1b82e4/volumes"
Oct 30 19:48:43.362952 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:43.362918    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w5-e9de60-85ffddb78-6zw8w"]
Oct 30 19:48:43.404686 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:43.404645    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w5-e9de60-75896ff6bc-vw4jn"]
Oct 30 19:48:45.697111 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:45.697079    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w5-e9de60-fwccr"]
Oct 30 19:48:45.709670 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:45.709643    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w5-e9de60-fwccr"]
Oct 30 19:48:46.287476 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:46.287427    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="90aa7fc8-86b9-48fe-a573-78780f3503cd" path="/var/lib/kubelet/pods/90aa7fc8-86b9-48fe-a573-78780f3503cd/volumes"
Oct 30 19:48:47.500631 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:48:47.500585    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:48:48.363351 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:48.363290    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w5-e9de60-85ffddb78-6zw8w" podUID="fb7394c6-8f11-4b08-954e-778579e3ec54" containerName="payload-processing" containerID="cri-o://ba86bbda5a0ab25ce6f597ad03f09b6e40162e442ed4c75316e4dfbdc3c5d817" gracePeriod=25
Oct 30 19:48:48.405915 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:48.405850    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-e9de60-75896ff6bc-vw4jn" podUID="efc8d0b8-a0b1-4c98-be74-87a9fcde2b96" containerName="payload-pre-processing" containerID="cri-o://d0dd28f211716e79c46f11d6cb847d89ab63379311a64285d6062347a66b4ec5" gracePeriod=25
Oct 30 19:48:48.514822 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:48.514787    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:48:48.769329 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:48.769294    5907 generic.go:356] "Generic (PLEG): container finished" podID="efc8d0b8-a0b1-4c98-be74-87a9fcde2b96" containerID="d0dd28f211716e79c46f11d6cb847d89ab63379311a64285d6062347a66b4ec5" exitCode=0
Oct 30 19:48:48.769459 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:48.769427    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-e9de60-75896ff6bc-vw4jn" event={"ID":"efc8d0b8-a0b1-4c98-be74-87a9fcde2b96","Type":"ContainerDied","Data":"d0dd28f211716e79c46f11d6cb847d89ab63379311a64285d6062347a66b4ec5"}
Oct 30 19:48:48.769528 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:48.769496    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-e9de60-75896ff6bc-vw4jn" event={"ID":"efc8d0b8-a0b1-4c98-be74-87a9fcde2b96","Type":"ContainerDied","Data":"95813788716eb1465b0b1f4d715befba439c88d264d8991439d73889d9ca9b0c"}
Oct 30 19:48:48.769528 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:48.769522    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="95813788716eb1465b0b1f4d715befba439c88d264d8991439d73889d9ca9b0c"
Oct 30 19:48:48.771280 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:48.771253    5907 generic.go:356] "Generic (PLEG): container finished" podID="fb7394c6-8f11-4b08-954e-778579e3ec54" containerID="ba86bbda5a0ab25ce6f597ad03f09b6e40162e442ed4c75316e4dfbdc3c5d817" exitCode=0
Oct 30 19:48:48.771386 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:48.771330    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w5-e9de60-85ffddb78-6zw8w" event={"ID":"fb7394c6-8f11-4b08-954e-778579e3ec54","Type":"ContainerDied","Data":"ba86bbda5a0ab25ce6f597ad03f09b6e40162e442ed4c75316e4dfbdc3c5d817"}
Oct 30 19:48:48.771463 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:48.771385    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w5-e9de60-85ffddb78-6zw8w" event={"ID":"fb7394c6-8f11-4b08-954e-778579e3ec54","Type":"ContainerDied","Data":"a37fdd0faccb87517827609acf9706473fb8c72f9849b4be344bcc946e621d46"}
Oct 30 19:48:48.771463 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:48.771409    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a37fdd0faccb87517827609acf9706473fb8c72f9849b4be344bcc946e621d46"
Oct 30 19:48:48.779693 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:48.779674    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w5-e9de60-85ffddb78-6zw8w"
Oct 30 19:48:48.783075 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:48.783055    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-e9de60-75896ff6bc-vw4jn"
Oct 30 19:48:48.928040 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:48.927964    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/efc8d0b8-a0b1-4c98-be74-87a9fcde2b96-kube-api-access-p5kdq\" (UniqueName: \"kubernetes.io/projected/efc8d0b8-a0b1-4c98-be74-87a9fcde2b96-kube-api-access-p5kdq\") pod \"efc8d0b8-a0b1-4c98-be74-87a9fcde2b96\" (UID: \"efc8d0b8-a0b1-4c98-be74-87a9fcde2b96\") "
Oct 30 19:48:48.928040 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:48.927999    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fb7394c6-8f11-4b08-954e-778579e3ec54-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb7394c6-8f11-4b08-954e-778579e3ec54-plugins-config-volume\") pod \"fb7394c6-8f11-4b08-954e-778579e3ec54\" (UID: \"fb7394c6-8f11-4b08-954e-778579e3ec54\") "
Oct 30 19:48:48.928288 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:48.928153    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fb7394c6-8f11-4b08-954e-778579e3ec54-kube-api-access-7n6js\" (UniqueName: \"kubernetes.io/projected/fb7394c6-8f11-4b08-954e-778579e3ec54-kube-api-access-7n6js\") pod \"fb7394c6-8f11-4b08-954e-778579e3ec54\" (UID: \"fb7394c6-8f11-4b08-954e-778579e3ec54\") "
Oct 30 19:48:48.928288 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:48.928185    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/efc8d0b8-a0b1-4c98-be74-87a9fcde2b96-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efc8d0b8-a0b1-4c98-be74-87a9fcde2b96-plugins-config-volume\") pod \"efc8d0b8-a0b1-4c98-be74-87a9fcde2b96\" (UID: \"efc8d0b8-a0b1-4c98-be74-87a9fcde2b96\") "
Oct 30 19:48:48.928520 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:48.928490    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb7394c6-8f11-4b08-954e-778579e3ec54-plugins-config-volume" pod "fb7394c6-8f11-4b08-954e-778579e3ec54" (UID: "fb7394c6-8f11-4b08-954e-778579e3ec54"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:48:48.928821 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:48.928535    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efc8d0b8-a0b1-4c98-be74-87a9fcde2b96-plugins-config-volume" pod "efc8d0b8-a0b1-4c98-be74-87a9fcde2b96" (UID: "efc8d0b8-a0b1-4c98-be74-87a9fcde2b96"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:48:48.928821 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:48.928601    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb7394c6-8f11-4b08-954e-778579e3ec54-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:48.930479 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:48.930452    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb7394c6-8f11-4b08-954e-778579e3ec54-kube-api-access-7n6js" pod "fb7394c6-8f11-4b08-954e-778579e3ec54" (UID: "fb7394c6-8f11-4b08-954e-778579e3ec54"). InnerVolumeSpecName "kube-api-access-7n6js". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:48.930591 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:48.930573    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efc8d0b8-a0b1-4c98-be74-87a9fcde2b96-kube-api-access-p5kdq" pod "efc8d0b8-a0b1-4c98-be74-87a9fcde2b96" (UID: "efc8d0b8-a0b1-4c98-be74-87a9fcde2b96"). InnerVolumeSpecName "kube-api-access-p5kdq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:49.030356 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:49.030325    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7n6js\" (UniqueName: \"kubernetes.io/projected/fb7394c6-8f11-4b08-954e-778579e3ec54-kube-api-access-7n6js\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:49.030356 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:49.030355    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efc8d0b8-a0b1-4c98-be74-87a9fcde2b96-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:49.030583 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:49.030368    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5kdq\" (UniqueName: \"kubernetes.io/projected/efc8d0b8-a0b1-4c98-be74-87a9fcde2b96-kube-api-access-p5kdq\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:49.777800 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:49.777765    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w5-e9de60-85ffddb78-6zw8w"
Oct 30 19:48:49.777800 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:49.777790    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-e9de60-75896ff6bc-vw4jn"
Oct 30 19:48:49.833179 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:49.833145    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w5-e9de60-75896ff6bc-vw4jn"]
Oct 30 19:48:49.837887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:49.837838    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w5-e9de60-75896ff6bc-vw4jn"]
Oct 30 19:48:49.882781 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:49.882751    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w5-e9de60-85ffddb78-6zw8w"]
Oct 30 19:48:49.888905 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:49.888877    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w5-e9de60-85ffddb78-6zw8w"]
Oct 30 19:48:50.287070 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:50.287037    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="efc8d0b8-a0b1-4c98-be74-87a9fcde2b96" path="/var/lib/kubelet/pods/efc8d0b8-a0b1-4c98-be74-87a9fcde2b96/volumes"
Oct 30 19:48:50.287438 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:50.287426    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fb7394c6-8f11-4b08-954e-778579e3ec54" path="/var/lib/kubelet/pods/fb7394c6-8f11-4b08-954e-778579e3ec54/volumes"
Oct 30 19:48:51.030128 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:51.030092    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-28885407-b2tx8"]
Oct 30 19:48:51.042606 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:51.042576    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-28885407-b2tx8"]
Oct 30 19:48:52.286990 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:52.286953    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eb6373d5-f4d9-4110-b9f9-2e1968021aee" path="/var/lib/kubelet/pods/eb6373d5-f4d9-4110-b9f9-2e1968021aee/volumes"
Oct 30 19:48:53.325959 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:53.325921    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"]
Oct 30 19:48:53.326498 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:53.326153    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9" podUID="ec8c99fb-671c-4aea-b30e-543bc0e3d6c1" containerName="istio-proxy" containerID="cri-o://5587a38ceec70e314bc2ec1f0bcf07aadb8e043b79045e68681402543bd319ff" gracePeriod=30
Oct 30 19:48:54.074980 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:54.074940    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-544fc8-bbbjd"]
Oct 30 19:48:54.075494 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:54.075476    5907 cpu_manager.go:414] "containerMap: removing container" podUID="6662d5b3-a2a2-4187-ae74-afaeaa1b82e4" containerName="istio-proxy"
Oct 30 19:48:54.075603 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:54.075497    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6662d5b3-a2a2-4187-ae74-afaeaa1b82e4" containerName="istio-proxy"
Oct 30 19:48:54.075603 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:54.075520    5907 cpu_manager.go:414] "containerMap: removing container" podUID="efc8d0b8-a0b1-4c98-be74-87a9fcde2b96" containerName="payload-pre-processing"
Oct 30 19:48:54.075603 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:54.075528    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="efc8d0b8-a0b1-4c98-be74-87a9fcde2b96" containerName="payload-pre-processing"
Oct 30 19:48:54.075603 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:54.075543    5907 cpu_manager.go:414] "containerMap: removing container" podUID="6662d5b3-a2a2-4187-ae74-afaeaa1b82e4" containerName="istio-proxy"
Oct 30 19:48:54.075603 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:54.075551    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6662d5b3-a2a2-4187-ae74-afaeaa1b82e4" containerName="istio-proxy"
Oct 30 19:48:54.075603 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:54.075586    5907 cpu_manager.go:414] "containerMap: removing container" podUID="fb7394c6-8f11-4b08-954e-778579e3ec54" containerName="payload-processing"
Oct 30 19:48:54.075603 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:54.075593    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fb7394c6-8f11-4b08-954e-778579e3ec54" containerName="payload-processing"
Oct 30 19:48:54.076002 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:54.075648    5907 cpu_manager.go:414] "containerMap: removing container" podUID="90aa7fc8-86b9-48fe-a573-78780f3503cd" containerName="revoke-keys"
Oct 30 19:48:54.076002 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:54.075657    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="90aa7fc8-86b9-48fe-a573-78780f3503cd" containerName="revoke-keys"
Oct 30 19:48:54.076002 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:54.075777    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="90aa7fc8-86b9-48fe-a573-78780f3503cd" containerName="revoke-keys"
Oct 30 19:48:54.076002 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:54.075792    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="6662d5b3-a2a2-4187-ae74-afaeaa1b82e4" containerName="istio-proxy"
Oct 30 19:48:54.076002 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:54.075802    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="efc8d0b8-a0b1-4c98-be74-87a9fcde2b96" containerName="payload-pre-processing"
Oct 30 19:48:54.076002 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:54.075816    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="fb7394c6-8f11-4b08-954e-778579e3ec54" containerName="payload-processing"
Oct 30 19:48:54.080739 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:54.080718    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-544fc8-bbbjd"
Oct 30 19:48:54.099625 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:54.099595    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-544fc8-bbbjd"]
Oct 30 19:48:54.125218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:54.125184    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:48:54.125414 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:54.125238    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:48:54.191408 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:54.191373    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dd233fe0-b16a-4dd2-9c1f-159da37c0a68-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w5-544fc8-bbbjd\" (UID: \"dd233fe0-b16a-4dd2-9c1f-159da37c0a68\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-544fc8-bbbjd"
Oct 30 19:48:54.191612 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:54.191436    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dd233fe0-b16a-4dd2-9c1f-159da37c0a68-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w5-544fc8-bbbjd\" (UID: \"dd233fe0-b16a-4dd2-9c1f-159da37c0a68\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-544fc8-bbbjd"
Oct 30 19:48:54.191612 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:54.191564    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkw7m\" (UniqueName: \"kubernetes.io/projected/dd233fe0-b16a-4dd2-9c1f-159da37c0a68-kube-api-access-hkw7m\") pod \"maas-api-revoke-keys-e2e-worker-w5-544fc8-bbbjd\" (UID: \"dd233fe0-b16a-4dd2-9c1f-159da37c0a68\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-544fc8-bbbjd"
Oct 30 19:48:54.292403 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:54.292373    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dd233fe0-b16a-4dd2-9c1f-159da37c0a68-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w5-544fc8-bbbjd\" (UID: \"dd233fe0-b16a-4dd2-9c1f-159da37c0a68\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-544fc8-bbbjd"
Oct 30 19:48:54.292588 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:54.292432    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dd233fe0-b16a-4dd2-9c1f-159da37c0a68-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w5-544fc8-bbbjd\" (UID: \"dd233fe0-b16a-4dd2-9c1f-159da37c0a68\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-544fc8-bbbjd"
Oct 30 19:48:54.292588 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:54.292518    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkw7m\" (UniqueName: \"kubernetes.io/projected/dd233fe0-b16a-4dd2-9c1f-159da37c0a68-kube-api-access-hkw7m\") pod \"maas-api-revoke-keys-e2e-worker-w5-544fc8-bbbjd\" (UID: \"dd233fe0-b16a-4dd2-9c1f-159da37c0a68\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-544fc8-bbbjd"
Oct 30 19:48:54.293076 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:54.293057    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dd233fe0-b16a-4dd2-9c1f-159da37c0a68-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w5-544fc8-bbbjd\" (UID: \"dd233fe0-b16a-4dd2-9c1f-159da37c0a68\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-544fc8-bbbjd"
Oct 30 19:48:54.294961 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:54.294936    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dd233fe0-b16a-4dd2-9c1f-159da37c0a68-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w5-544fc8-bbbjd\" (UID: \"dd233fe0-b16a-4dd2-9c1f-159da37c0a68\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-544fc8-bbbjd"
Oct 30 19:48:54.314800 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:54.314776    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkw7m\" (UniqueName: \"kubernetes.io/projected/dd233fe0-b16a-4dd2-9c1f-159da37c0a68-kube-api-access-hkw7m\") pod \"maas-api-revoke-keys-e2e-worker-w5-544fc8-bbbjd\" (UID: \"dd233fe0-b16a-4dd2-9c1f-159da37c0a68\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-544fc8-bbbjd"
Oct 30 19:48:54.395979 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:54.395951    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-544fc8-bbbjd"
Oct 30 19:48:54.701279 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:54.701252    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-544fc8-bbbjd"]
Oct 30 19:48:54.703066 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:48:54.703042    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd233fe0_b16a_4dd2_9c1f_159da37c0a68.slice/crio-371f2088d3283999a5e855e0e69869dccb6ba732cc25999c07bf5cff730084a9 WatchSource:0}: Error finding container 371f2088d3283999a5e855e0e69869dccb6ba732cc25999c07bf5cff730084a9: Status 404 returned error can't find the container with id 371f2088d3283999a5e855e0e69869dccb6ba732cc25999c07bf5cff730084a9
Oct 30 19:48:54.829843 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:54.829805    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-544fc8-bbbjd" event={"ID":"dd233fe0-b16a-4dd2-9c1f-159da37c0a68","Type":"ContainerStarted","Data":"371f2088d3283999a5e855e0e69869dccb6ba732cc25999c07bf5cff730084a9"}
Oct 30 19:48:55.837009 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:55.836974    5907 patch_prober.go:29] interesting pod/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:48:55.837483 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:55.837037    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9" podUID="ec8c99fb-671c-4aea-b30e-543bc0e3d6c1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:48:55.838618 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:55.838590    5907 generic.go:356] "Generic (PLEG): container finished" podID="dd233fe0-b16a-4dd2-9c1f-159da37c0a68" containerID="09107fde4b9e170b8c3ff6c6828f8d8596ed2a5fa8dd14620be3c06bb3e456f8" exitCode=0
Oct 30 19:48:55.838724 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:55.838650    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-544fc8-bbbjd" event={"ID":"dd233fe0-b16a-4dd2-9c1f-159da37c0a68","Type":"ContainerDied","Data":"09107fde4b9e170b8c3ff6c6828f8d8596ed2a5fa8dd14620be3c06bb3e456f8"}
Oct 30 19:48:57.005721 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:57.005671    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-544fc8-bbbjd"
Oct 30 19:48:57.121852 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:57.121825    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dd233fe0-b16a-4dd2-9c1f-159da37c0a68-kube-api-access-hkw7m\" (UniqueName: \"kubernetes.io/projected/dd233fe0-b16a-4dd2-9c1f-159da37c0a68-kube-api-access-hkw7m\") pod \"dd233fe0-b16a-4dd2-9c1f-159da37c0a68\" (UID: \"dd233fe0-b16a-4dd2-9c1f-159da37c0a68\") "
Oct 30 19:48:57.122119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:57.122054    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dd233fe0-b16a-4dd2-9c1f-159da37c0a68-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dd233fe0-b16a-4dd2-9c1f-159da37c0a68-maas-api-xks-ca\") pod \"dd233fe0-b16a-4dd2-9c1f-159da37c0a68\" (UID: \"dd233fe0-b16a-4dd2-9c1f-159da37c0a68\") "
Oct 30 19:48:57.122119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:57.122089    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dd233fe0-b16a-4dd2-9c1f-159da37c0a68-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dd233fe0-b16a-4dd2-9c1f-159da37c0a68-maas-api-openshift-ca\") pod \"dd233fe0-b16a-4dd2-9c1f-159da37c0a68\" (UID: \"dd233fe0-b16a-4dd2-9c1f-159da37c0a68\") "
Oct 30 19:48:57.122567 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:57.122543    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd233fe0-b16a-4dd2-9c1f-159da37c0a68-maas-api-openshift-ca" pod "dd233fe0-b16a-4dd2-9c1f-159da37c0a68" (UID: "dd233fe0-b16a-4dd2-9c1f-159da37c0a68"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:48:57.124099 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:57.124066    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd233fe0-b16a-4dd2-9c1f-159da37c0a68-kube-api-access-hkw7m" pod "dd233fe0-b16a-4dd2-9c1f-159da37c0a68" (UID: "dd233fe0-b16a-4dd2-9c1f-159da37c0a68"). InnerVolumeSpecName "kube-api-access-hkw7m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:57.124227 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:57.124133    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd233fe0-b16a-4dd2-9c1f-159da37c0a68-maas-api-xks-ca" pod "dd233fe0-b16a-4dd2-9c1f-159da37c0a68" (UID: "dd233fe0-b16a-4dd2-9c1f-159da37c0a68"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:48:57.223690 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:57.223648    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkw7m\" (UniqueName: \"kubernetes.io/projected/dd233fe0-b16a-4dd2-9c1f-159da37c0a68-kube-api-access-hkw7m\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:57.223690 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:57.223687    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dd233fe0-b16a-4dd2-9c1f-159da37c0a68-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:57.223690 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:57.223696    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dd233fe0-b16a-4dd2-9c1f-159da37c0a68-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:57.857819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:57.857784    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-544fc8-bbbjd"
Oct 30 19:48:57.858036 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:57.857834    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-544fc8-bbbjd" event={"ID":"dd233fe0-b16a-4dd2-9c1f-159da37c0a68","Type":"ContainerDied","Data":"371f2088d3283999a5e855e0e69869dccb6ba732cc25999c07bf5cff730084a9"}
Oct 30 19:48:57.858036 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:57.857894    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="371f2088d3283999a5e855e0e69869dccb6ba732cc25999c07bf5cff730084a9"
Oct 30 19:48:58.624549 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.624243    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:48:58.739973 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.739936    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-istiod-ca-cert\") pod \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") "
Oct 30 19:48:58.740167 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.740040    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-istio-envoy\") pod \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") "
Oct 30 19:48:58.740167 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.740064    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-workload-certs\") pod \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") "
Oct 30 19:48:58.740167 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.740101    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-istio-podinfo\") pod \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") "
Oct 30 19:48:58.740167 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.740138    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-istio-data\") pod \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") "
Oct 30 19:48:58.740398 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.740200    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-credential-socket\") pod \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") "
Oct 30 19:48:58.740398 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.740254    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-istio-token\" (UniqueName: \"kubernetes.io/projected/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-istio-token\") pod \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") "
Oct 30 19:48:58.740398 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.740282    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-workload-socket\") pod \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") "
Oct 30 19:48:58.740398 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.740313    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-kube-api-access-74jdc\" (UniqueName: \"kubernetes.io/projected/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-kube-api-access-74jdc\") pod \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\" (UID: \"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1\") "
Oct 30 19:48:58.740398 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.740336    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-istiod-ca-cert" pod "ec8c99fb-671c-4aea-b30e-543bc0e3d6c1" (UID: "ec8c99fb-671c-4aea-b30e-543bc0e3d6c1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:48:58.740728 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.740391    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-workload-certs" pod "ec8c99fb-671c-4aea-b30e-543bc0e3d6c1" (UID: "ec8c99fb-671c-4aea-b30e-543bc0e3d6c1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:58.740728 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.740451    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-istio-data" pod "ec8c99fb-671c-4aea-b30e-543bc0e3d6c1" (UID: "ec8c99fb-671c-4aea-b30e-543bc0e3d6c1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:58.740728 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.740517    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-credential-socket" pod "ec8c99fb-671c-4aea-b30e-543bc0e3d6c1" (UID: "ec8c99fb-671c-4aea-b30e-543bc0e3d6c1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:58.740728 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.740633    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-workload-socket" pod "ec8c99fb-671c-4aea-b30e-543bc0e3d6c1" (UID: "ec8c99fb-671c-4aea-b30e-543bc0e3d6c1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:58.740931 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.740809    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:58.740931 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.740826    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:58.740931 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.740836    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:58.740931 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.740845    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:58.740931 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.740853    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:58.742390 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.742365    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-istio-podinfo" pod "ec8c99fb-671c-4aea-b30e-543bc0e3d6c1" (UID: "ec8c99fb-671c-4aea-b30e-543bc0e3d6c1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:48:58.742791 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.742764    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-istio-token" pod "ec8c99fb-671c-4aea-b30e-543bc0e3d6c1" (UID: "ec8c99fb-671c-4aea-b30e-543bc0e3d6c1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:58.743049 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.743029    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-istio-envoy" pod "ec8c99fb-671c-4aea-b30e-543bc0e3d6c1" (UID: "ec8c99fb-671c-4aea-b30e-543bc0e3d6c1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:58.743158 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.743121    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-kube-api-access-74jdc" pod "ec8c99fb-671c-4aea-b30e-543bc0e3d6c1" (UID: "ec8c99fb-671c-4aea-b30e-543bc0e3d6c1"). InnerVolumeSpecName "kube-api-access-74jdc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:58.841657 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.841620    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:58.841657 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.841654    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:58.841657 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.841663    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:58.842218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.841674    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74jdc\" (UniqueName: \"kubernetes.io/projected/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1-kube-api-access-74jdc\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:48:58.865831 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.865801    5907 generic.go:356] "Generic (PLEG): container finished" podID="ec8c99fb-671c-4aea-b30e-543bc0e3d6c1" containerID="5587a38ceec70e314bc2ec1f0bcf07aadb8e043b79045e68681402543bd319ff" exitCode=0
Oct 30 19:48:58.866013 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.865891    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9" event={"ID":"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1","Type":"ContainerDied","Data":"5587a38ceec70e314bc2ec1f0bcf07aadb8e043b79045e68681402543bd319ff"}
Oct 30 19:48:58.866013 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.865923    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"
Oct 30 19:48:58.866013 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.865959    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9" event={"ID":"ec8c99fb-671c-4aea-b30e-543bc0e3d6c1","Type":"ContainerDied","Data":"0f58c4b18b51544f3ee0f03522c1037cd75611c35cc616f7ca39ea3859209cc8"}
Oct 30 19:48:58.866013 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.865984    5907 scope.go:122] "RemoveContainer" containerID="5587a38ceec70e314bc2ec1f0bcf07aadb8e043b79045e68681402543bd319ff"
Oct 30 19:48:58.876649 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.876630    5907 scope.go:122] "RemoveContainer" containerID="b37856698cdd9aa3a02cf54ddf8eaacf36ce6bd93693522ce0733e0b689dda9e"
Oct 30 19:48:58.886100 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.886083    5907 scope.go:122] "RemoveContainer" containerID="5587a38ceec70e314bc2ec1f0bcf07aadb8e043b79045e68681402543bd319ff"
Oct 30 19:48:58.886353 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:48:58.886336    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5587a38ceec70e314bc2ec1f0bcf07aadb8e043b79045e68681402543bd319ff\": container with ID starting with 5587a38ceec70e314bc2ec1f0bcf07aadb8e043b79045e68681402543bd319ff not found: ID does not exist" containerID="5587a38ceec70e314bc2ec1f0bcf07aadb8e043b79045e68681402543bd319ff"
Oct 30 19:48:58.886407 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.886361    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5587a38ceec70e314bc2ec1f0bcf07aadb8e043b79045e68681402543bd319ff"} err="failed to get container status \"5587a38ceec70e314bc2ec1f0bcf07aadb8e043b79045e68681402543bd319ff\": rpc error: code = NotFound desc = could not find container \"5587a38ceec70e314bc2ec1f0bcf07aadb8e043b79045e68681402543bd319ff\": container with ID starting with 5587a38ceec70e314bc2ec1f0bcf07aadb8e043b79045e68681402543bd319ff not found: ID does not exist"
Oct 30 19:48:58.886407 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.886380    5907 scope.go:122] "RemoveContainer" containerID="b37856698cdd9aa3a02cf54ddf8eaacf36ce6bd93693522ce0733e0b689dda9e"
Oct 30 19:48:58.886590 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:48:58.886567    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b37856698cdd9aa3a02cf54ddf8eaacf36ce6bd93693522ce0733e0b689dda9e\": container with ID starting with b37856698cdd9aa3a02cf54ddf8eaacf36ce6bd93693522ce0733e0b689dda9e not found: ID does not exist" containerID="b37856698cdd9aa3a02cf54ddf8eaacf36ce6bd93693522ce0733e0b689dda9e"
Oct 30 19:48:58.886630 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.886598    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b37856698cdd9aa3a02cf54ddf8eaacf36ce6bd93693522ce0733e0b689dda9e"} err="failed to get container status \"b37856698cdd9aa3a02cf54ddf8eaacf36ce6bd93693522ce0733e0b689dda9e\": rpc error: code = NotFound desc = could not find container \"b37856698cdd9aa3a02cf54ddf8eaacf36ce6bd93693522ce0733e0b689dda9e\": container with ID starting with b37856698cdd9aa3a02cf54ddf8eaacf36ce6bd93693522ce0733e0b689dda9e not found: ID does not exist"
Oct 30 19:48:58.954483 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.954452    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"]
Oct 30 19:48:58.968043 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:48:58.968014    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w5-e9de60-openshift-default-5bd476d9cc-5j2r9"]
Oct 30 19:49:00.287204 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:00.287172    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ec8c99fb-671c-4aea-b30e-543bc0e3d6c1" path="/var/lib/kubelet/pods/ec8c99fb-671c-4aea-b30e-543bc0e3d6c1/volumes"
Oct 30 19:49:03.094486 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:03.094453    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"]
Oct 30 19:49:03.094950 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:03.094654    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv" podUID="685bf818-6ca9-4903-a55e-b4255a635543" containerName="istio-proxy" containerID="cri-o://76727cafec64590f0731e737e2b2d35789ffb5868da2e6f2fef735ca2e6ef694" gracePeriod=30
Oct 30 19:49:03.972339 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:03.972301    5907 patch_prober.go:29] interesting pod/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:49:03.972529 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:03.972380    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv" podUID="685bf818-6ca9-4903-a55e-b4255a635543" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:49:04.483633 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:04.483586    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-544fc8-78bc5bc8bd-wjdrd"]
Oct 30 19:49:04.517049 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:04.517011    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-544fc8-cb5b6cbd7-h7tjd"]
Oct 30 19:49:05.908274 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:05.908237    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7762c4d1-6rr8x"]
Oct 30 19:49:05.908747 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:05.908732    5907 cpu_manager.go:414] "containerMap: removing container" podUID="dd233fe0-b16a-4dd2-9c1f-159da37c0a68" containerName="revoke-keys"
Oct 30 19:49:05.908797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:05.908748    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dd233fe0-b16a-4dd2-9c1f-159da37c0a68" containerName="revoke-keys"
Oct 30 19:49:05.908853 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:05.908807    5907 cpu_manager.go:414] "containerMap: removing container" podUID="ec8c99fb-671c-4aea-b30e-543bc0e3d6c1" containerName="istio-proxy"
Oct 30 19:49:05.908853 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:05.908813    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ec8c99fb-671c-4aea-b30e-543bc0e3d6c1" containerName="istio-proxy"
Oct 30 19:49:05.908853 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:05.908844    5907 cpu_manager.go:414] "containerMap: removing container" podUID="ec8c99fb-671c-4aea-b30e-543bc0e3d6c1" containerName="istio-proxy"
Oct 30 19:49:05.909048 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:05.908853    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ec8c99fb-671c-4aea-b30e-543bc0e3d6c1" containerName="istio-proxy"
Oct 30 19:49:05.909048 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:05.909035    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="ec8c99fb-671c-4aea-b30e-543bc0e3d6c1" containerName="istio-proxy"
Oct 30 19:49:05.909143 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:05.909051    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="dd233fe0-b16a-4dd2-9c1f-159da37c0a68" containerName="revoke-keys"
Oct 30 19:49:05.909143 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:05.909066    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="6662d5b3-a2a2-4187-ae74-afaeaa1b82e4" containerName="istio-proxy"
Oct 30 19:49:05.913400 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:05.913377    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7762c4d1-6rr8x"
Oct 30 19:49:05.935394 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:05.935365    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7762c4d1-6rr8x"]
Oct 30 19:49:05.945516 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:05.945488    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:49:05.945677 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:05.945543    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:49:06.017109 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:06.017063    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7762c4d1-6rr8x\" (UID: \"4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7762c4d1-6rr8x"
Oct 30 19:49:06.017289 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:06.017227    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4rkm\" (UniqueName: \"kubernetes.io/projected/4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4-kube-api-access-g4rkm\") pod \"maas-api-revoke-sub-7762c4d1-6rr8x\" (UID: \"4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7762c4d1-6rr8x"
Oct 30 19:49:06.017289 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:06.017266    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7762c4d1-6rr8x\" (UID: \"4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7762c4d1-6rr8x"
Oct 30 19:49:06.118370 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:06.118334    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7762c4d1-6rr8x\" (UID: \"4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7762c4d1-6rr8x"
Oct 30 19:49:06.118561 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:06.118432    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4rkm\" (UniqueName: \"kubernetes.io/projected/4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4-kube-api-access-g4rkm\") pod \"maas-api-revoke-sub-7762c4d1-6rr8x\" (UID: \"4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7762c4d1-6rr8x"
Oct 30 19:49:06.118561 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:06.118458    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7762c4d1-6rr8x\" (UID: \"4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7762c4d1-6rr8x"
Oct 30 19:49:06.118990 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:06.118969    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7762c4d1-6rr8x\" (UID: \"4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7762c4d1-6rr8x"
Oct 30 19:49:06.120975 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:06.120958    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7762c4d1-6rr8x\" (UID: \"4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7762c4d1-6rr8x"
Oct 30 19:49:06.139705 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:06.139679    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4rkm\" (UniqueName: \"kubernetes.io/projected/4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4-kube-api-access-g4rkm\") pod \"maas-api-revoke-sub-7762c4d1-6rr8x\" (UID: \"4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7762c4d1-6rr8x"
Oct 30 19:49:06.226605 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:06.226536    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7762c4d1-6rr8x"
Oct 30 19:49:06.541464 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:06.541435    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7762c4d1-6rr8x"]
Oct 30 19:49:06.542559 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:49:06.542532    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bd53c00_d91a_4fc5_9a1d_8626bdbec4e4.slice/crio-5bdb5368d541769a71929cf202c3ff598a7db158edd82a1b1e8246b717afbc8a WatchSource:0}: Error finding container 5bdb5368d541769a71929cf202c3ff598a7db158edd82a1b1e8246b717afbc8a: Status 404 returned error can't find the container with id 5bdb5368d541769a71929cf202c3ff598a7db158edd82a1b1e8246b717afbc8a
Oct 30 19:49:06.925110 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:06.925075    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7762c4d1-6rr8x" event={"ID":"4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4","Type":"ContainerStarted","Data":"53014dabaab60ab459b86dedef3147560d5696e6f53bf67a2dd0c0b9e12138ed"}
Oct 30 19:49:06.925517 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:06.925137    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7762c4d1-6rr8x" event={"ID":"4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4","Type":"ContainerStarted","Data":"5bdb5368d541769a71929cf202c3ff598a7db158edd82a1b1e8246b717afbc8a"}
Oct 30 19:49:06.970358 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:06.970306    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7762c4d1-6rr8x" podStartSLOduration=1.9702909960000001 podStartE2EDuration="1.970290996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:49:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:49:06.969485703 +0000 UTC m=+1658.812417644" watchObservedRunningTime="2026-10-30 19:49:06.970290996 +0000 UTC m=+1658.813222965"
Oct 30 19:49:07.934376 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:07.934346    5907 generic.go:356] "Generic (PLEG): container finished" podID="4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4" containerID="53014dabaab60ab459b86dedef3147560d5696e6f53bf67a2dd0c0b9e12138ed" exitCode=0
Oct 30 19:49:07.934774 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:07.934432    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7762c4d1-6rr8x" event={"ID":"4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4","Type":"ContainerDied","Data":"53014dabaab60ab459b86dedef3147560d5696e6f53bf67a2dd0c0b9e12138ed"}
Oct 30 19:49:08.376250 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:08.376215    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"
Oct 30 19:49:08.441484 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:08.441452    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/685bf818-6ca9-4903-a55e-b4255a635543-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/685bf818-6ca9-4903-a55e-b4255a635543-istio-podinfo\") pod \"685bf818-6ca9-4903-a55e-b4255a635543\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") "
Oct 30 19:49:08.441768 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:08.441748    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/685bf818-6ca9-4903-a55e-b4255a635543-istio-token\" (UniqueName: \"kubernetes.io/projected/685bf818-6ca9-4903-a55e-b4255a635543-istio-token\") pod \"685bf818-6ca9-4903-a55e-b4255a635543\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") "
Oct 30 19:49:08.441959 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:08.441940    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/685bf818-6ca9-4903-a55e-b4255a635543-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/685bf818-6ca9-4903-a55e-b4255a635543-istio-envoy\") pod \"685bf818-6ca9-4903-a55e-b4255a635543\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") "
Oct 30 19:49:08.442137 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:08.442118    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/685bf818-6ca9-4903-a55e-b4255a635543-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/685bf818-6ca9-4903-a55e-b4255a635543-workload-socket\") pod \"685bf818-6ca9-4903-a55e-b4255a635543\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") "
Oct 30 19:49:08.442439 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:08.442391    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/685bf818-6ca9-4903-a55e-b4255a635543-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/685bf818-6ca9-4903-a55e-b4255a635543-istiod-ca-cert\") pod \"685bf818-6ca9-4903-a55e-b4255a635543\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") "
Oct 30 19:49:08.442665 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:08.442505    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/685bf818-6ca9-4903-a55e-b4255a635543-workload-socket" pod "685bf818-6ca9-4903-a55e-b4255a635543" (UID: "685bf818-6ca9-4903-a55e-b4255a635543"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:08.442835 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:08.442807    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/685bf818-6ca9-4903-a55e-b4255a635543-istiod-ca-cert" pod "685bf818-6ca9-4903-a55e-b4255a635543" (UID: "685bf818-6ca9-4903-a55e-b4255a635543"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:49:08.443017 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:08.442988    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/685bf818-6ca9-4903-a55e-b4255a635543-workload-certs" pod "685bf818-6ca9-4903-a55e-b4255a635543" (UID: "685bf818-6ca9-4903-a55e-b4255a635543"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:08.443017 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:08.442608    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/685bf818-6ca9-4903-a55e-b4255a635543-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/685bf818-6ca9-4903-a55e-b4255a635543-workload-certs\") pod \"685bf818-6ca9-4903-a55e-b4255a635543\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") "
Oct 30 19:49:08.443215 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:08.443067    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/685bf818-6ca9-4903-a55e-b4255a635543-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/685bf818-6ca9-4903-a55e-b4255a635543-istio-data\") pod \"685bf818-6ca9-4903-a55e-b4255a635543\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") "
Oct 30 19:49:08.443215 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:08.443131    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/685bf818-6ca9-4903-a55e-b4255a635543-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/685bf818-6ca9-4903-a55e-b4255a635543-credential-socket\") pod \"685bf818-6ca9-4903-a55e-b4255a635543\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") "
Oct 30 19:49:08.443215 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:08.443169    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/685bf818-6ca9-4903-a55e-b4255a635543-kube-api-access-97hqk\" (UniqueName: \"kubernetes.io/projected/685bf818-6ca9-4903-a55e-b4255a635543-kube-api-access-97hqk\") pod \"685bf818-6ca9-4903-a55e-b4255a635543\" (UID: \"685bf818-6ca9-4903-a55e-b4255a635543\") "
Oct 30 19:49:08.443446 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:08.443424    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/685bf818-6ca9-4903-a55e-b4255a635543-credential-socket" pod "685bf818-6ca9-4903-a55e-b4255a635543" (UID: "685bf818-6ca9-4903-a55e-b4255a635543"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:08.443591 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:08.443576    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/685bf818-6ca9-4903-a55e-b4255a635543-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:49:08.443664 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:08.443594    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/685bf818-6ca9-4903-a55e-b4255a635543-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:49:08.443664 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:08.443606    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/685bf818-6ca9-4903-a55e-b4255a635543-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:49:08.443664 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:08.443622    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/685bf818-6ca9-4903-a55e-b4255a635543-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:49:08.443831 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:08.443675    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/685bf818-6ca9-4903-a55e-b4255a635543-istio-data" pod "685bf818-6ca9-4903-a55e-b4255a635543" (UID: "685bf818-6ca9-4903-a55e-b4255a635543"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:08.446564 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:08.445267    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/685bf818-6ca9-4903-a55e-b4255a635543-istio-podinfo" pod "685bf818-6ca9-4903-a55e-b4255a635543" (UID: "685bf818-6ca9-4903-a55e-b4255a635543"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:49:08.446564 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:08.445286    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/685bf818-6ca9-4903-a55e-b4255a635543-istio-envoy" pod "685bf818-6ca9-4903-a55e-b4255a635543" (UID: "685bf818-6ca9-4903-a55e-b4255a635543"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:08.446990 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:08.446961    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/685bf818-6ca9-4903-a55e-b4255a635543-istio-token" pod "685bf818-6ca9-4903-a55e-b4255a635543" (UID: "685bf818-6ca9-4903-a55e-b4255a635543"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:49:08.450643 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:08.450617    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/685bf818-6ca9-4903-a55e-b4255a635543-kube-api-access-97hqk" pod "685bf818-6ca9-4903-a55e-b4255a635543" (UID: "685bf818-6ca9-4903-a55e-b4255a635543"). InnerVolumeSpecName "kube-api-access-97hqk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:49:08.544940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:08.544737    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/685bf818-6ca9-4903-a55e-b4255a635543-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:49:08.544940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:08.544775    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97hqk\" (UniqueName: \"kubernetes.io/projected/685bf818-6ca9-4903-a55e-b4255a635543-kube-api-access-97hqk\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:49:08.544940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:08.544789    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/685bf818-6ca9-4903-a55e-b4255a635543-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:49:08.544940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:08.544802    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/685bf818-6ca9-4903-a55e-b4255a635543-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:49:08.544940 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:08.544815    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/685bf818-6ca9-4903-a55e-b4255a635543-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:49:08.941992 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:08.941954    5907 generic.go:356] "Generic (PLEG): container finished" podID="685bf818-6ca9-4903-a55e-b4255a635543" containerID="76727cafec64590f0731e737e2b2d35789ffb5868da2e6f2fef735ca2e6ef694" exitCode=0
Oct 30 19:49:08.942417 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:08.942011    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"
Oct 30 19:49:08.942417 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:08.942042    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv" event={"ID":"685bf818-6ca9-4903-a55e-b4255a635543","Type":"ContainerDied","Data":"76727cafec64590f0731e737e2b2d35789ffb5868da2e6f2fef735ca2e6ef694"}
Oct 30 19:49:08.942417 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:08.942103    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv" event={"ID":"685bf818-6ca9-4903-a55e-b4255a635543","Type":"ContainerDied","Data":"6615373276e95c735ff4bbb56d15b97bd1642588d832fceb46d6a9cce8129fea"}
Oct 30 19:49:08.942417 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:08.942126    5907 scope.go:122] "RemoveContainer" containerID="76727cafec64590f0731e737e2b2d35789ffb5868da2e6f2fef735ca2e6ef694"
Oct 30 19:49:08.957822 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:08.957800    5907 scope.go:122] "RemoveContainer" containerID="76727cafec64590f0731e737e2b2d35789ffb5868da2e6f2fef735ca2e6ef694"
Oct 30 19:49:08.958268 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:49:08.958221    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76727cafec64590f0731e737e2b2d35789ffb5868da2e6f2fef735ca2e6ef694\": container with ID starting with 76727cafec64590f0731e737e2b2d35789ffb5868da2e6f2fef735ca2e6ef694 not found: ID does not exist" containerID="76727cafec64590f0731e737e2b2d35789ffb5868da2e6f2fef735ca2e6ef694"
Oct 30 19:49:08.958268 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:08.958255    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76727cafec64590f0731e737e2b2d35789ffb5868da2e6f2fef735ca2e6ef694"} err="failed to get container status \"76727cafec64590f0731e737e2b2d35789ffb5868da2e6f2fef735ca2e6ef694\": rpc error: code = NotFound desc = could not find container \"76727cafec64590f0731e737e2b2d35789ffb5868da2e6f2fef735ca2e6ef694\": container with ID starting with 76727cafec64590f0731e737e2b2d35789ffb5868da2e6f2fef735ca2e6ef694 not found: ID does not exist"
Oct 30 19:49:09.002886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:09.002837    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"]
Oct 30 19:49:09.017086 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:09.017057    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-k7cmv"]
Oct 30 19:49:09.085574 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:09.085543    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7762c4d1-6rr8x"
Oct 30 19:49:09.152176 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:09.152148    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4-kube-api-access-g4rkm\" (UniqueName: \"kubernetes.io/projected/4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4-kube-api-access-g4rkm\") pod \"4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4\" (UID: \"4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4\") "
Oct 30 19:49:09.152366 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:09.152205    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4-maas-api-xks-ca\") pod \"4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4\" (UID: \"4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4\") "
Oct 30 19:49:09.152366 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:09.152314    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4-maas-api-openshift-ca\") pod \"4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4\" (UID: \"4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4\") "
Oct 30 19:49:09.152661 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:09.152632    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4-maas-api-openshift-ca" pod "4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4" (UID: "4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:49:09.152797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:09.152762    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:49:09.154516 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:09.154491    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4-kube-api-access-g4rkm" pod "4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4" (UID: "4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4"). InnerVolumeSpecName "kube-api-access-g4rkm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:49:09.154516 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:09.154510    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4-maas-api-xks-ca" pod "4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4" (UID: "4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:49:09.253979 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:09.253910    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4rkm\" (UniqueName: \"kubernetes.io/projected/4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4-kube-api-access-g4rkm\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:49:09.253979 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:09.253934    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:49:09.484810 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:09.484754    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w5-544fc8-78bc5bc8bd-wjdrd" podUID="615c6e69-747b-4294-ae5b-e0958527002c" containerName="payload-processing" containerID="cri-o://3403130a8c3692d69fa1926b8a30593c925449d9819d4695a00ac86607e73e44" gracePeriod=25
Oct 30 19:49:09.518370 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:09.518267    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-544fc8-cb5b6cbd7-h7tjd" podUID="97367526-d21f-45f0-bc3f-d36ae4a1fe7c" containerName="payload-pre-processing" containerID="cri-o://88c377fab912723f7bc47ba8f7f4be48d4fc6b75a265068abf827009a1c23094" gracePeriod=25
Oct 30 19:49:09.949762 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:09.949727    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7762c4d1-6rr8x"
Oct 30 19:49:09.950246 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:09.949757    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7762c4d1-6rr8x" event={"ID":"4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4","Type":"ContainerDied","Data":"5bdb5368d541769a71929cf202c3ff598a7db158edd82a1b1e8246b717afbc8a"}
Oct 30 19:49:09.950246 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:09.949813    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5bdb5368d541769a71929cf202c3ff598a7db158edd82a1b1e8246b717afbc8a"
Oct 30 19:49:10.287722 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:10.287641    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="685bf818-6ca9-4903-a55e-b4255a635543" path="/var/lib/kubelet/pods/685bf818-6ca9-4903-a55e-b4255a635543/volumes"
Oct 30 19:49:15.013353 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:15.013309    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w5-544fc8-78bc5bc8bd-wjdrd" podUID="615c6e69-747b-4294-ae5b-e0958527002c" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:49:15.013353 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:15.013338    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-544fc8-cb5b6cbd7-h7tjd" podUID="97367526-d21f-45f0-bc3f-d36ae4a1fe7c" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:49:17.500298 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:49:17.500259    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:49:18.984981 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:18.984946    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:49:21.612035 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:21.611998    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-544fc8-bbbjd"]
Oct 30 19:49:21.632364 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:21.632330    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-544fc8-bbbjd"]
Oct 30 19:49:22.286886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:22.286821    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dd233fe0-b16a-4dd2-9c1f-159da37c0a68" path="/var/lib/kubelet/pods/dd233fe0-b16a-4dd2-9c1f-159da37c0a68/volumes"
Oct 30 19:49:25.013955 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:25.013919    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w5-544fc8-78bc5bc8bd-wjdrd" podUID="615c6e69-747b-4294-ae5b-e0958527002c" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:49:25.014385 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:25.013919    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-544fc8-cb5b6cbd7-h7tjd" podUID="97367526-d21f-45f0-bc3f-d36ae4a1fe7c" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:49:25.031852 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:25.031821    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dbba9287-sx94j"]
Oct 30 19:49:25.044687 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:25.044663    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dbba9287-sx94j"]
Oct 30 19:49:26.288111 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:26.288079    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8a1d7fdf-a22a-4803-91c2-d756ef33f542" path="/var/lib/kubelet/pods/8a1d7fdf-a22a-4803-91c2-d756ef33f542/volumes"
Oct 30 19:49:27.028625 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:27.028591    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-05420cec-nxtf7"]
Oct 30 19:49:27.041988 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:27.041959    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6c23f404-mm85l"]
Oct 30 19:49:27.043907 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:27.043888    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-05420cec-nxtf7"]
Oct 30 19:49:27.045762 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:27.045740    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6c23f404-mm85l"]
Oct 30 19:49:28.288173 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:28.288141    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="05d43121-612e-4afa-afbd-37776f632f55" path="/var/lib/kubelet/pods/05d43121-612e-4afa-afbd-37776f632f55/volumes"
Oct 30 19:49:28.288654 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:28.288542    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0c8a3d58-6f3e-43ef-9adf-910ad6504e87" path="/var/lib/kubelet/pods/0c8a3d58-6f3e-43ef-9adf-910ad6504e87/volumes"
Oct 30 19:49:28.602788 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:28.602712    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"]
Oct 30 19:49:28.602988 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:28.602961    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc" podUID="e440f6c0-de4c-4a55-af89-7ea0552419a9" containerName="istio-proxy" containerID="cri-o://22f20b47048fa05eb29274c8c9834003bc3152b321c440cbcfb3da19a574c460" gracePeriod=30
Oct 30 19:49:29.771635 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:29.771602    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-544fc8-78bc5bc8bd-wjdrd"
Oct 30 19:49:29.811632 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:29.811608    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-544fc8-cb5b6cbd7-h7tjd"
Oct 30 19:49:29.875810 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:29.875778    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/615c6e69-747b-4294-ae5b-e0958527002c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/615c6e69-747b-4294-ae5b-e0958527002c-plugins-config-volume\") pod \"615c6e69-747b-4294-ae5b-e0958527002c\" (UID: \"615c6e69-747b-4294-ae5b-e0958527002c\") "
Oct 30 19:49:29.876035 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:29.875826    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/97367526-d21f-45f0-bc3f-d36ae4a1fe7c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97367526-d21f-45f0-bc3f-d36ae4a1fe7c-plugins-config-volume\") pod \"97367526-d21f-45f0-bc3f-d36ae4a1fe7c\" (UID: \"97367526-d21f-45f0-bc3f-d36ae4a1fe7c\") "
Oct 30 19:49:29.876035 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:29.875967    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/615c6e69-747b-4294-ae5b-e0958527002c-kube-api-access-jprj8\" (UniqueName: \"kubernetes.io/projected/615c6e69-747b-4294-ae5b-e0958527002c-kube-api-access-jprj8\") pod \"615c6e69-747b-4294-ae5b-e0958527002c\" (UID: \"615c6e69-747b-4294-ae5b-e0958527002c\") "
Oct 30 19:49:29.876035 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:29.876001    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/97367526-d21f-45f0-bc3f-d36ae4a1fe7c-kube-api-access-vz59m\" (UniqueName: \"kubernetes.io/projected/97367526-d21f-45f0-bc3f-d36ae4a1fe7c-kube-api-access-vz59m\") pod \"97367526-d21f-45f0-bc3f-d36ae4a1fe7c\" (UID: \"97367526-d21f-45f0-bc3f-d36ae4a1fe7c\") "
Oct 30 19:49:29.876295 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:29.876272    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/615c6e69-747b-4294-ae5b-e0958527002c-plugins-config-volume" pod "615c6e69-747b-4294-ae5b-e0958527002c" (UID: "615c6e69-747b-4294-ae5b-e0958527002c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:49:29.876295 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:29.876278    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97367526-d21f-45f0-bc3f-d36ae4a1fe7c-plugins-config-volume" pod "97367526-d21f-45f0-bc3f-d36ae4a1fe7c" (UID: "97367526-d21f-45f0-bc3f-d36ae4a1fe7c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:49:29.876451 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:29.876369    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/615c6e69-747b-4294-ae5b-e0958527002c-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:49:29.876451 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:29.876387    5907 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97367526-d21f-45f0-bc3f-d36ae4a1fe7c-plugins-config-volume\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:49:29.878245 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:29.878214    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/615c6e69-747b-4294-ae5b-e0958527002c-kube-api-access-jprj8" pod "615c6e69-747b-4294-ae5b-e0958527002c" (UID: "615c6e69-747b-4294-ae5b-e0958527002c"). InnerVolumeSpecName "kube-api-access-jprj8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:49:29.878359 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:29.878314    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97367526-d21f-45f0-bc3f-d36ae4a1fe7c-kube-api-access-vz59m" pod "97367526-d21f-45f0-bc3f-d36ae4a1fe7c" (UID: "97367526-d21f-45f0-bc3f-d36ae4a1fe7c"). InnerVolumeSpecName "kube-api-access-vz59m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:49:29.976847 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:29.976819    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jprj8\" (UniqueName: \"kubernetes.io/projected/615c6e69-747b-4294-ae5b-e0958527002c-kube-api-access-jprj8\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:49:29.976847 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:29.976844    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vz59m\" (UniqueName: \"kubernetes.io/projected/97367526-d21f-45f0-bc3f-d36ae4a1fe7c-kube-api-access-vz59m\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:49:30.083775 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:30.083700    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:49:30.098463 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:30.098434    5907 generic.go:356] "Generic (PLEG): container finished" podID="97367526-d21f-45f0-bc3f-d36ae4a1fe7c" containerID="88c377fab912723f7bc47ba8f7f4be48d4fc6b75a265068abf827009a1c23094" exitCode=0
Oct 30 19:49:30.098582 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:30.098483    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-544fc8-cb5b6cbd7-h7tjd"
Oct 30 19:49:30.098582 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:30.098516    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-544fc8-cb5b6cbd7-h7tjd" event={"ID":"97367526-d21f-45f0-bc3f-d36ae4a1fe7c","Type":"ContainerDied","Data":"88c377fab912723f7bc47ba8f7f4be48d4fc6b75a265068abf827009a1c23094"}
Oct 30 19:49:30.098582 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:30.098565    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-544fc8-cb5b6cbd7-h7tjd" event={"ID":"97367526-d21f-45f0-bc3f-d36ae4a1fe7c","Type":"ContainerDied","Data":"bdf318f6875fb1c9ea525769ee1aeb658b844409c15aa17e3650c26b97f405d5"}
Oct 30 19:49:30.098805 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:30.098586    5907 scope.go:122] "RemoveContainer" containerID="88c377fab912723f7bc47ba8f7f4be48d4fc6b75a265068abf827009a1c23094"
Oct 30 19:49:30.100276 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:30.100253    5907 generic.go:356] "Generic (PLEG): container finished" podID="615c6e69-747b-4294-ae5b-e0958527002c" containerID="3403130a8c3692d69fa1926b8a30593c925449d9819d4695a00ac86607e73e44" exitCode=0
Oct 30 19:49:30.100369 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:30.100307    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-544fc8-78bc5bc8bd-wjdrd"
Oct 30 19:49:30.100369 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:30.100333    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-544fc8-78bc5bc8bd-wjdrd" event={"ID":"615c6e69-747b-4294-ae5b-e0958527002c","Type":"ContainerDied","Data":"3403130a8c3692d69fa1926b8a30593c925449d9819d4695a00ac86607e73e44"}
Oct 30 19:49:30.100484 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:30.100381    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-544fc8-78bc5bc8bd-wjdrd" event={"ID":"615c6e69-747b-4294-ae5b-e0958527002c","Type":"ContainerDied","Data":"8a9b4e0d1c69c496eff5a07ae98b46b7ddf0018707e8f7ab9892ef3c1698bcd2"}
Oct 30 19:49:30.109111 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:30.109094    5907 scope.go:122] "RemoveContainer" containerID="88c377fab912723f7bc47ba8f7f4be48d4fc6b75a265068abf827009a1c23094"
Oct 30 19:49:30.109408 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:49:30.109387    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88c377fab912723f7bc47ba8f7f4be48d4fc6b75a265068abf827009a1c23094\": container with ID starting with 88c377fab912723f7bc47ba8f7f4be48d4fc6b75a265068abf827009a1c23094 not found: ID does not exist" containerID="88c377fab912723f7bc47ba8f7f4be48d4fc6b75a265068abf827009a1c23094"
Oct 30 19:49:30.109503 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:30.109419    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88c377fab912723f7bc47ba8f7f4be48d4fc6b75a265068abf827009a1c23094"} err="failed to get container status \"88c377fab912723f7bc47ba8f7f4be48d4fc6b75a265068abf827009a1c23094\": rpc error: code = NotFound desc = could not find container \"88c377fab912723f7bc47ba8f7f4be48d4fc6b75a265068abf827009a1c23094\": container with ID starting with 88c377fab912723f7bc47ba8f7f4be48d4fc6b75a265068abf827009a1c23094 not found: ID does not exist"
Oct 30 19:49:30.109503 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:30.109443    5907 scope.go:122] "RemoveContainer" containerID="3403130a8c3692d69fa1926b8a30593c925449d9819d4695a00ac86607e73e44"
Oct 30 19:49:30.118780 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:30.118761    5907 scope.go:122] "RemoveContainer" containerID="3403130a8c3692d69fa1926b8a30593c925449d9819d4695a00ac86607e73e44"
Oct 30 19:49:30.119079 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:49:30.119059    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3403130a8c3692d69fa1926b8a30593c925449d9819d4695a00ac86607e73e44\": container with ID starting with 3403130a8c3692d69fa1926b8a30593c925449d9819d4695a00ac86607e73e44 not found: ID does not exist" containerID="3403130a8c3692d69fa1926b8a30593c925449d9819d4695a00ac86607e73e44"
Oct 30 19:49:30.119139 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:30.119085    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3403130a8c3692d69fa1926b8a30593c925449d9819d4695a00ac86607e73e44"} err="failed to get container status \"3403130a8c3692d69fa1926b8a30593c925449d9819d4695a00ac86607e73e44\": rpc error: code = NotFound desc = could not find container \"3403130a8c3692d69fa1926b8a30593c925449d9819d4695a00ac86607e73e44\": container with ID starting with 3403130a8c3692d69fa1926b8a30593c925449d9819d4695a00ac86607e73e44 not found: ID does not exist"
Oct 30 19:49:30.169361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:30.169331    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-544fc8-78bc5bc8bd-wjdrd"]
Oct 30 19:49:30.187731 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:30.187703    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-544fc8-78bc5bc8bd-wjdrd"]
Oct 30 19:49:30.237516 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:30.237485    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-544fc8-cb5b6cbd7-h7tjd"]
Oct 30 19:49:30.244237 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:30.244215    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-544fc8-cb5b6cbd7-h7tjd"]
Oct 30 19:49:30.286954 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:30.286921    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="615c6e69-747b-4294-ae5b-e0958527002c" path="/var/lib/kubelet/pods/615c6e69-747b-4294-ae5b-e0958527002c/volumes"
Oct 30 19:49:30.287317 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:30.287304    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="97367526-d21f-45f0-bc3f-d36ae4a1fe7c" path="/var/lib/kubelet/pods/97367526-d21f-45f0-bc3f-d36ae4a1fe7c/volumes"
Oct 30 19:49:33.446185 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:33.446149    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b9b54ffb7-7xdf5"]
Oct 30 19:49:33.446687 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:33.446363    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-6b9b54ffb7-7xdf5" podUID="6be54382-a07f-4e83-949e-3bf550c46599" containerName="manager" containerID="cri-o://5f24650652fa11ac5c910eeed31d6a994f3f8d5eb6b7da503b3172529bbc46ec" gracePeriod=30
Oct 30 19:49:33.763608 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:33.761160    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b9b54ffb7-7xdf5"
Oct 30 19:49:33.931568 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:33.931540    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6be54382-a07f-4e83-949e-3bf550c46599-kube-api-access-dfh7c\" (UniqueName: \"kubernetes.io/projected/6be54382-a07f-4e83-949e-3bf550c46599-kube-api-access-dfh7c\") pod \"6be54382-a07f-4e83-949e-3bf550c46599\" (UID: \"6be54382-a07f-4e83-949e-3bf550c46599\") "
Oct 30 19:49:33.931781 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:33.931601    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6be54382-a07f-4e83-949e-3bf550c46599-webhook-cert\" (UniqueName: \"kubernetes.io/secret/6be54382-a07f-4e83-949e-3bf550c46599-webhook-cert\") pod \"6be54382-a07f-4e83-949e-3bf550c46599\" (UID: \"6be54382-a07f-4e83-949e-3bf550c46599\") "
Oct 30 19:49:33.931781 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:33.931687    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6be54382-a07f-4e83-949e-3bf550c46599-metrics-certs\" (UniqueName: \"kubernetes.io/secret/6be54382-a07f-4e83-949e-3bf550c46599-metrics-certs\") pod \"6be54382-a07f-4e83-949e-3bf550c46599\" (UID: \"6be54382-a07f-4e83-949e-3bf550c46599\") "
Oct 30 19:49:33.948328 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:33.948279    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6be54382-a07f-4e83-949e-3bf550c46599-kube-api-access-dfh7c" pod "6be54382-a07f-4e83-949e-3bf550c46599" (UID: "6be54382-a07f-4e83-949e-3bf550c46599"). InnerVolumeSpecName "kube-api-access-dfh7c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:49:33.951371 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:33.951338    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6be54382-a07f-4e83-949e-3bf550c46599-webhook-cert" pod "6be54382-a07f-4e83-949e-3bf550c46599" (UID: "6be54382-a07f-4e83-949e-3bf550c46599"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:49:33.951822 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:33.951796    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6be54382-a07f-4e83-949e-3bf550c46599-metrics-certs" pod "6be54382-a07f-4e83-949e-3bf550c46599" (UID: "6be54382-a07f-4e83-949e-3bf550c46599"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:49:33.975884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:33.975292    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"
Oct 30 19:49:34.054560 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.053123    5907 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6be54382-a07f-4e83-949e-3bf550c46599-webhook-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:49:34.054560 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.053156    5907 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6be54382-a07f-4e83-949e-3bf550c46599-metrics-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:49:34.054560 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.053171    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfh7c\" (UniqueName: \"kubernetes.io/projected/6be54382-a07f-4e83-949e-3bf550c46599-kube-api-access-dfh7c\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:49:34.056970 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.056911    5907 scope.go:122] "RemoveContainer" containerID="ba86bbda5a0ab25ce6f597ad03f09b6e40162e442ed4c75316e4dfbdc3c5d817"
Oct 30 19:49:34.073468 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.073444    5907 scope.go:122] "RemoveContainer" containerID="d0dd28f211716e79c46f11d6cb847d89ab63379311a64285d6062347a66b4ec5"
Oct 30 19:49:34.089147 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.089051    5907 scope.go:122] "RemoveContainer" containerID="ae076d042a9aaba941e1df68930a7d698d4945bd5f02feb795f79659f49c778f"
Oct 30 19:49:34.100204 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.100181    5907 scope.go:122] "RemoveContainer" containerID="5f24650652fa11ac5c910eeed31d6a994f3f8d5eb6b7da503b3172529bbc46ec"
Oct 30 19:49:34.112681 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.112659    5907 scope.go:122] "RemoveContainer" containerID="22f20b47048fa05eb29274c8c9834003bc3152b321c440cbcfb3da19a574c460"
Oct 30 19:49:34.125026 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.124638    5907 scope.go:122] "RemoveContainer" containerID="97672ebd1e21daf3833a9e603d443d42fa5fd4c7c4ff1b01b8c3f0e7779d0923"
Oct 30 19:49:34.132187 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.132151    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b9b54ffb7-7xdf5" event={"ID":"6be54382-a07f-4e83-949e-3bf550c46599","Type":"ContainerDied","Data":"5f24650652fa11ac5c910eeed31d6a994f3f8d5eb6b7da503b3172529bbc46ec"}
Oct 30 19:49:34.132335 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.132263    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b9b54ffb7-7xdf5" event={"ID":"6be54382-a07f-4e83-949e-3bf550c46599","Type":"ContainerDied","Data":"1761863f841944c032f0dfbab5550af9462b03e66237a9d6a969ba86e3e882ba"}
Oct 30 19:49:34.132413 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.132352    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b9b54ffb7-7xdf5"
Oct 30 19:49:34.151159 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.150965    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc" event={"ID":"e440f6c0-de4c-4a55-af89-7ea0552419a9","Type":"ContainerDied","Data":"22f20b47048fa05eb29274c8c9834003bc3152b321c440cbcfb3da19a574c460"}
Oct 30 19:49:34.151159 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.151033    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc" event={"ID":"e440f6c0-de4c-4a55-af89-7ea0552419a9","Type":"ContainerDied","Data":"21e27caa1a1ddb62926ad77027416d38f21f8cebb9be07079f95996a6c160847"}
Oct 30 19:49:34.151159 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.150978    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"
Oct 30 19:49:34.154479 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.154454    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e440f6c0-de4c-4a55-af89-7ea0552419a9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e440f6c0-de4c-4a55-af89-7ea0552419a9-workload-socket\") pod \"e440f6c0-de4c-4a55-af89-7ea0552419a9\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") "
Oct 30 19:49:34.154573 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.154548    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e440f6c0-de4c-4a55-af89-7ea0552419a9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e440f6c0-de4c-4a55-af89-7ea0552419a9-istiod-ca-cert\") pod \"e440f6c0-de4c-4a55-af89-7ea0552419a9\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") "
Oct 30 19:49:34.154645 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.154586    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e440f6c0-de4c-4a55-af89-7ea0552419a9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e440f6c0-de4c-4a55-af89-7ea0552419a9-workload-certs\") pod \"e440f6c0-de4c-4a55-af89-7ea0552419a9\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") "
Oct 30 19:49:34.154645 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.154618    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e440f6c0-de4c-4a55-af89-7ea0552419a9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e440f6c0-de4c-4a55-af89-7ea0552419a9-istio-envoy\") pod \"e440f6c0-de4c-4a55-af89-7ea0552419a9\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") "
Oct 30 19:49:34.154762 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.154690    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e440f6c0-de4c-4a55-af89-7ea0552419a9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e440f6c0-de4c-4a55-af89-7ea0552419a9-istio-data\") pod \"e440f6c0-de4c-4a55-af89-7ea0552419a9\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") "
Oct 30 19:49:34.154762 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.154754    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e440f6c0-de4c-4a55-af89-7ea0552419a9-istio-token\" (UniqueName: \"kubernetes.io/projected/e440f6c0-de4c-4a55-af89-7ea0552419a9-istio-token\") pod \"e440f6c0-de4c-4a55-af89-7ea0552419a9\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") "
Oct 30 19:49:34.154901 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.154810    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e440f6c0-de4c-4a55-af89-7ea0552419a9-kube-api-access-7zmlf\" (UniqueName: \"kubernetes.io/projected/e440f6c0-de4c-4a55-af89-7ea0552419a9-kube-api-access-7zmlf\") pod \"e440f6c0-de4c-4a55-af89-7ea0552419a9\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") "
Oct 30 19:49:34.154967 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.154900    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e440f6c0-de4c-4a55-af89-7ea0552419a9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e440f6c0-de4c-4a55-af89-7ea0552419a9-credential-socket\") pod \"e440f6c0-de4c-4a55-af89-7ea0552419a9\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") "
Oct 30 19:49:34.154967 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.154961    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e440f6c0-de4c-4a55-af89-7ea0552419a9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e440f6c0-de4c-4a55-af89-7ea0552419a9-istio-podinfo\") pod \"e440f6c0-de4c-4a55-af89-7ea0552419a9\" (UID: \"e440f6c0-de4c-4a55-af89-7ea0552419a9\") "
Oct 30 19:49:34.155080 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.154963    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e440f6c0-de4c-4a55-af89-7ea0552419a9-workload-certs" pod "e440f6c0-de4c-4a55-af89-7ea0552419a9" (UID: "e440f6c0-de4c-4a55-af89-7ea0552419a9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:34.155239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.155221    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e440f6c0-de4c-4a55-af89-7ea0552419a9-istiod-ca-cert" pod "e440f6c0-de4c-4a55-af89-7ea0552419a9" (UID: "e440f6c0-de4c-4a55-af89-7ea0552419a9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:49:34.155378 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.155363    5907 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e440f6c0-de4c-4a55-af89-7ea0552419a9-istiod-ca-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:49:34.155448 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.155384    5907 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e440f6c0-de4c-4a55-af89-7ea0552419a9-workload-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:49:34.155717 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.155687    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e440f6c0-de4c-4a55-af89-7ea0552419a9-workload-socket" pod "e440f6c0-de4c-4a55-af89-7ea0552419a9" (UID: "e440f6c0-de4c-4a55-af89-7ea0552419a9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:34.156121 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.156087    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e440f6c0-de4c-4a55-af89-7ea0552419a9-istio-data" pod "e440f6c0-de4c-4a55-af89-7ea0552419a9" (UID: "e440f6c0-de4c-4a55-af89-7ea0552419a9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:34.156445 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.156425    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e440f6c0-de4c-4a55-af89-7ea0552419a9-credential-socket" pod "e440f6c0-de4c-4a55-af89-7ea0552419a9" (UID: "e440f6c0-de4c-4a55-af89-7ea0552419a9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:34.158737 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.158708    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e440f6c0-de4c-4a55-af89-7ea0552419a9-istio-token" pod "e440f6c0-de4c-4a55-af89-7ea0552419a9" (UID: "e440f6c0-de4c-4a55-af89-7ea0552419a9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:49:34.160555 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.160527    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e440f6c0-de4c-4a55-af89-7ea0552419a9-istio-envoy" pod "e440f6c0-de4c-4a55-af89-7ea0552419a9" (UID: "e440f6c0-de4c-4a55-af89-7ea0552419a9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:34.160555 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.160548    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e440f6c0-de4c-4a55-af89-7ea0552419a9-kube-api-access-7zmlf" pod "e440f6c0-de4c-4a55-af89-7ea0552419a9" (UID: "e440f6c0-de4c-4a55-af89-7ea0552419a9"). InnerVolumeSpecName "kube-api-access-7zmlf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:49:34.161195 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.161171    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e440f6c0-de4c-4a55-af89-7ea0552419a9-istio-podinfo" pod "e440f6c0-de4c-4a55-af89-7ea0552419a9" (UID: "e440f6c0-de4c-4a55-af89-7ea0552419a9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:49:34.215566 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.215539    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b9b54ffb7-7xdf5"]
Oct 30 19:49:34.222970 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.222947    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6b9b54ffb7-7xdf5"]
Oct 30 19:49:34.257095 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.257057    5907 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e440f6c0-de4c-4a55-af89-7ea0552419a9-workload-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:49:34.257095 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.257097    5907 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e440f6c0-de4c-4a55-af89-7ea0552419a9-istio-envoy\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:49:34.257095 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.257113    5907 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e440f6c0-de4c-4a55-af89-7ea0552419a9-istio-data\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:49:34.257407 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.257126    5907 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e440f6c0-de4c-4a55-af89-7ea0552419a9-istio-token\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:49:34.257407 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.257140    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zmlf\" (UniqueName: \"kubernetes.io/projected/e440f6c0-de4c-4a55-af89-7ea0552419a9-kube-api-access-7zmlf\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:49:34.257407 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.257154    5907 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e440f6c0-de4c-4a55-af89-7ea0552419a9-credential-socket\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:49:34.257407 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.257167    5907 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e440f6c0-de4c-4a55-af89-7ea0552419a9-istio-podinfo\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:49:34.291032 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.290981    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6be54382-a07f-4e83-949e-3bf550c46599" path="/var/lib/kubelet/pods/6be54382-a07f-4e83-949e-3bf550c46599/volumes"
Oct 30 19:49:34.496521 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.496478    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"]
Oct 30 19:49:34.509789 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.509742    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-544fc8-openshift-default-6b56b8b795-r6jwc"]
Oct 30 19:49:34.936886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.936315    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6b9b54ffb7-4zlnt"]
Oct 30 19:49:34.937102 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.936984    5907 cpu_manager.go:414] "containerMap: removing container" podUID="6be54382-a07f-4e83-949e-3bf550c46599" containerName="manager"
Oct 30 19:49:34.937102 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.937003    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6be54382-a07f-4e83-949e-3bf550c46599" containerName="manager"
Oct 30 19:49:34.937102 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.937067    5907 cpu_manager.go:414] "containerMap: removing container" podUID="97367526-d21f-45f0-bc3f-d36ae4a1fe7c" containerName="payload-pre-processing"
Oct 30 19:49:34.937102 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.937075    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="97367526-d21f-45f0-bc3f-d36ae4a1fe7c" containerName="payload-pre-processing"
Oct 30 19:49:34.937102 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.937093    5907 cpu_manager.go:414] "containerMap: removing container" podUID="615c6e69-747b-4294-ae5b-e0958527002c" containerName="payload-processing"
Oct 30 19:49:34.937102 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.937100    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="615c6e69-747b-4294-ae5b-e0958527002c" containerName="payload-processing"
Oct 30 19:49:34.937476 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.937117    5907 cpu_manager.go:414] "containerMap: removing container" podUID="4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4" containerName="revoke-keys"
Oct 30 19:49:34.937476 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.937124    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4" containerName="revoke-keys"
Oct 30 19:49:34.937476 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.937135    5907 cpu_manager.go:414] "containerMap: removing container" podUID="e440f6c0-de4c-4a55-af89-7ea0552419a9" containerName="istio-proxy"
Oct 30 19:49:34.937476 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.937142    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e440f6c0-de4c-4a55-af89-7ea0552419a9" containerName="istio-proxy"
Oct 30 19:49:34.937476 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.937152    5907 cpu_manager.go:414] "containerMap: removing container" podUID="685bf818-6ca9-4903-a55e-b4255a635543" containerName="istio-proxy"
Oct 30 19:49:34.937476 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.937159    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="685bf818-6ca9-4903-a55e-b4255a635543" containerName="istio-proxy"
Oct 30 19:49:34.937476 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.937261    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="97367526-d21f-45f0-bc3f-d36ae4a1fe7c" containerName="payload-pre-processing"
Oct 30 19:49:34.937476 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.937277    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="615c6e69-747b-4294-ae5b-e0958527002c" containerName="payload-processing"
Oct 30 19:49:34.937476 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.937286    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4" containerName="revoke-keys"
Oct 30 19:49:34.937476 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.937296    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="e440f6c0-de4c-4a55-af89-7ea0552419a9" containerName="istio-proxy"
Oct 30 19:49:34.937476 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.937304    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="685bf818-6ca9-4903-a55e-b4255a635543" containerName="istio-proxy"
Oct 30 19:49:34.937476 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.937313    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="ec8c99fb-671c-4aea-b30e-543bc0e3d6c1" containerName="istio-proxy"
Oct 30 19:49:34.937476 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.937321    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="6be54382-a07f-4e83-949e-3bf550c46599" containerName="manager"
Oct 30 19:49:34.943577 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.943553    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b9b54ffb7-4zlnt"
Oct 30 19:49:34.958884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.958121    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b9b54ffb7-4zlnt"]
Oct 30 19:49:34.971904 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.970854    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 19:49:34.971904 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.971083    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 19:49:34.971904 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.971216    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 19:49:34.971904 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.971333    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-58qcr\""
Oct 30 19:49:34.975884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.972956    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drn8n\" (UniqueName: \"kubernetes.io/projected/af954f65-b68b-44d7-9e78-b0c249f88f38-kube-api-access-drn8n\") pod \"maas-controller-6b9b54ffb7-4zlnt\" (UID: \"af954f65-b68b-44d7-9e78-b0c249f88f38\") " pod="opendatahub/maas-controller-6b9b54ffb7-4zlnt"
Oct 30 19:49:34.975884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.973046    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/af954f65-b68b-44d7-9e78-b0c249f88f38-webhook-cert\") pod \"maas-controller-6b9b54ffb7-4zlnt\" (UID: \"af954f65-b68b-44d7-9e78-b0c249f88f38\") " pod="opendatahub/maas-controller-6b9b54ffb7-4zlnt"
Oct 30 19:49:34.975884 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:34.973103    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af954f65-b68b-44d7-9e78-b0c249f88f38-metrics-certs\") pod \"maas-controller-6b9b54ffb7-4zlnt\" (UID: \"af954f65-b68b-44d7-9e78-b0c249f88f38\") " pod="opendatahub/maas-controller-6b9b54ffb7-4zlnt"
Oct 30 19:49:35.075888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:35.075487    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/af954f65-b68b-44d7-9e78-b0c249f88f38-webhook-cert\") pod \"maas-controller-6b9b54ffb7-4zlnt\" (UID: \"af954f65-b68b-44d7-9e78-b0c249f88f38\") " pod="opendatahub/maas-controller-6b9b54ffb7-4zlnt"
Oct 30 19:49:35.075888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:35.075570    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af954f65-b68b-44d7-9e78-b0c249f88f38-metrics-certs\") pod \"maas-controller-6b9b54ffb7-4zlnt\" (UID: \"af954f65-b68b-44d7-9e78-b0c249f88f38\") " pod="opendatahub/maas-controller-6b9b54ffb7-4zlnt"
Oct 30 19:49:35.075888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:35.075688    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drn8n\" (UniqueName: \"kubernetes.io/projected/af954f65-b68b-44d7-9e78-b0c249f88f38-kube-api-access-drn8n\") pod \"maas-controller-6b9b54ffb7-4zlnt\" (UID: \"af954f65-b68b-44d7-9e78-b0c249f88f38\") " pod="opendatahub/maas-controller-6b9b54ffb7-4zlnt"
Oct 30 19:49:35.134920 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:35.126523    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af954f65-b68b-44d7-9e78-b0c249f88f38-metrics-certs\") pod \"maas-controller-6b9b54ffb7-4zlnt\" (UID: \"af954f65-b68b-44d7-9e78-b0c249f88f38\") " pod="opendatahub/maas-controller-6b9b54ffb7-4zlnt"
Oct 30 19:49:35.141590 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:35.141513    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drn8n\" (UniqueName: \"kubernetes.io/projected/af954f65-b68b-44d7-9e78-b0c249f88f38-kube-api-access-drn8n\") pod \"maas-controller-6b9b54ffb7-4zlnt\" (UID: \"af954f65-b68b-44d7-9e78-b0c249f88f38\") " pod="opendatahub/maas-controller-6b9b54ffb7-4zlnt"
Oct 30 19:49:35.242944 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:35.208397    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/af954f65-b68b-44d7-9e78-b0c249f88f38-webhook-cert\") pod \"maas-controller-6b9b54ffb7-4zlnt\" (UID: \"af954f65-b68b-44d7-9e78-b0c249f88f38\") " pod="opendatahub/maas-controller-6b9b54ffb7-4zlnt"
Oct 30 19:49:35.325886 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:35.325061    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b9b54ffb7-4zlnt"
Oct 30 19:49:35.884888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:35.864978    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-9tdql container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:49:35.884888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:35.865058    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql" podUID="8db50859-98fc-49aa-806b-4fbed56b1d00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:49:36.276978 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:36.276849    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b9b54ffb7-4zlnt"]
Oct 30 19:49:36.346259 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:36.346231    5907 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:49:36.346638 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:36.346400    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e440f6c0-de4c-4a55-af89-7ea0552419a9" path="/var/lib/kubelet/pods/e440f6c0-de4c-4a55-af89-7ea0552419a9/volumes"
Oct 30 19:49:37.252832 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:37.252799    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b9b54ffb7-4zlnt" event={"ID":"af954f65-b68b-44d7-9e78-b0c249f88f38","Type":"ContainerStarted","Data":"3b79a8440912ade663a89822a3af2d5683941621deea5ccf3012ea756e04a981"}
Oct 30 19:49:37.253271 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:37.252871    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b9b54ffb7-4zlnt" event={"ID":"af954f65-b68b-44d7-9e78-b0c249f88f38","Type":"ContainerStarted","Data":"f7821276c705c1a04dba4a26cd547b77e7998b84ffa68d967517571c390a6120"}
Oct 30 19:49:37.253271 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:37.252905    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6b9b54ffb7-4zlnt"
Oct 30 19:49:37.296708 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:37.296658    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-6b9b54ffb7-4zlnt" podStartSLOduration=2.8068403870000003 podStartE2EDuration="3.296642408s" totalImagesPullingTime="489.802021ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:49:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:49:37.295994076 +0000 UTC m=+1689.138926034" watchObservedRunningTime="2026-10-30 19:49:37.296642408 +0000 UTC m=+1689.139574341"
Oct 30 19:49:47.499154 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:49:47.499113    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:49:48.260817 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:48.260789    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6b9b54ffb7-4zlnt"
Oct 30 19:49:57.201689 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:57.201650    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-lmn6m"]
Oct 30 19:49:57.205533 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:57.205512    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-lmn6m"
Oct 30 19:49:57.222769 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:57.222745    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:49:57.236132 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:57.236104    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:49:57.238416 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:57.238388    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-lmn6m"]
Oct 30 19:49:57.298312 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:57.298278    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/56409091-d2f5-43ab-9412-d95605c5e3b4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-85237e45-lmn6m\" (UID: \"56409091-d2f5-43ab-9412-d95605c5e3b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-lmn6m"
Oct 30 19:49:57.298469 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:57.298333    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/56409091-d2f5-43ab-9412-d95605c5e3b4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-85237e45-lmn6m\" (UID: \"56409091-d2f5-43ab-9412-d95605c5e3b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-lmn6m"
Oct 30 19:49:57.298469 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:57.298354    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzf5h\" (UniqueName: \"kubernetes.io/projected/56409091-d2f5-43ab-9412-d95605c5e3b4-kube-api-access-pzf5h\") pod \"maas-api-revoke-sub-85237e45-lmn6m\" (UID: \"56409091-d2f5-43ab-9412-d95605c5e3b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-lmn6m"
Oct 30 19:49:57.400230 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:57.400191    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/56409091-d2f5-43ab-9412-d95605c5e3b4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-85237e45-lmn6m\" (UID: \"56409091-d2f5-43ab-9412-d95605c5e3b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-lmn6m"
Oct 30 19:49:57.400398 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:57.400261    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/56409091-d2f5-43ab-9412-d95605c5e3b4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-85237e45-lmn6m\" (UID: \"56409091-d2f5-43ab-9412-d95605c5e3b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-lmn6m"
Oct 30 19:49:57.400398 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:57.400290    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzf5h\" (UniqueName: \"kubernetes.io/projected/56409091-d2f5-43ab-9412-d95605c5e3b4-kube-api-access-pzf5h\") pod \"maas-api-revoke-sub-85237e45-lmn6m\" (UID: \"56409091-d2f5-43ab-9412-d95605c5e3b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-lmn6m"
Oct 30 19:49:57.400818 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:57.400794    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/56409091-d2f5-43ab-9412-d95605c5e3b4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-85237e45-lmn6m\" (UID: \"56409091-d2f5-43ab-9412-d95605c5e3b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-lmn6m"
Oct 30 19:49:57.402710 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:57.402684    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/56409091-d2f5-43ab-9412-d95605c5e3b4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-85237e45-lmn6m\" (UID: \"56409091-d2f5-43ab-9412-d95605c5e3b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-lmn6m"
Oct 30 19:49:57.421432 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:57.421403    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzf5h\" (UniqueName: \"kubernetes.io/projected/56409091-d2f5-43ab-9412-d95605c5e3b4-kube-api-access-pzf5h\") pod \"maas-api-revoke-sub-85237e45-lmn6m\" (UID: \"56409091-d2f5-43ab-9412-d95605c5e3b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-lmn6m"
Oct 30 19:49:57.518748 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:57.518680    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-lmn6m"
Oct 30 19:49:57.819727 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:57.819703    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-lmn6m"]
Oct 30 19:49:57.822002 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:49:57.821971    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56409091_d2f5_43ab_9412_d95605c5e3b4.slice/crio-61fd3091248e1ed7d7b064e0354f1c6e0b0a370ce4085666170635f6dd9d13c2 WatchSource:0}: Error finding container 61fd3091248e1ed7d7b064e0354f1c6e0b0a370ce4085666170635f6dd9d13c2: Status 404 returned error can't find the container with id 61fd3091248e1ed7d7b064e0354f1c6e0b0a370ce4085666170635f6dd9d13c2
Oct 30 19:49:58.380496 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:58.380410    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-lmn6m" event={"ID":"56409091-d2f5-43ab-9412-d95605c5e3b4","Type":"ContainerStarted","Data":"f462dcc69551f0c49096736df39cfba06acfc583855f8ca0f518bd9e2d554bd5"}
Oct 30 19:49:58.380496 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:58.380476    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-lmn6m" event={"ID":"56409091-d2f5-43ab-9412-d95605c5e3b4","Type":"ContainerStarted","Data":"61fd3091248e1ed7d7b064e0354f1c6e0b0a370ce4085666170635f6dd9d13c2"}
Oct 30 19:49:58.425793 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:58.425724    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-lmn6m" podStartSLOduration=1.425689587 podStartE2EDuration="1.425689587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:49:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:49:58.425195329 +0000 UTC m=+1710.268127269" watchObservedRunningTime="2026-10-30 19:49:58.425689587 +0000 UTC m=+1710.268621526"
Oct 30 19:49:59.388399 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:59.388367    5907 generic.go:356] "Generic (PLEG): container finished" podID="56409091-d2f5-43ab-9412-d95605c5e3b4" containerID="f462dcc69551f0c49096736df39cfba06acfc583855f8ca0f518bd9e2d554bd5" exitCode=0
Oct 30 19:49:59.388917 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:49:59.388463    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-lmn6m" event={"ID":"56409091-d2f5-43ab-9412-d95605c5e3b4","Type":"ContainerDied","Data":"f462dcc69551f0c49096736df39cfba06acfc583855f8ca0f518bd9e2d554bd5"}
Oct 30 19:50:00.541312 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:00.541290    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-lmn6m"
Oct 30 19:50:00.650431 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:00.650349    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/56409091-d2f5-43ab-9412-d95605c5e3b4-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/56409091-d2f5-43ab-9412-d95605c5e3b4-maas-api-xks-ca\") pod \"56409091-d2f5-43ab-9412-d95605c5e3b4\" (UID: \"56409091-d2f5-43ab-9412-d95605c5e3b4\") "
Oct 30 19:50:00.650431 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:00.650405    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/56409091-d2f5-43ab-9412-d95605c5e3b4-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/56409091-d2f5-43ab-9412-d95605c5e3b4-maas-api-openshift-ca\") pod \"56409091-d2f5-43ab-9412-d95605c5e3b4\" (UID: \"56409091-d2f5-43ab-9412-d95605c5e3b4\") "
Oct 30 19:50:00.650669 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:00.650506    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/56409091-d2f5-43ab-9412-d95605c5e3b4-kube-api-access-pzf5h\" (UniqueName: \"kubernetes.io/projected/56409091-d2f5-43ab-9412-d95605c5e3b4-kube-api-access-pzf5h\") pod \"56409091-d2f5-43ab-9412-d95605c5e3b4\" (UID: \"56409091-d2f5-43ab-9412-d95605c5e3b4\") "
Oct 30 19:50:00.650791 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:00.650764    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56409091-d2f5-43ab-9412-d95605c5e3b4-maas-api-openshift-ca" pod "56409091-d2f5-43ab-9412-d95605c5e3b4" (UID: "56409091-d2f5-43ab-9412-d95605c5e3b4"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:50:00.650918 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:00.650904    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/56409091-d2f5-43ab-9412-d95605c5e3b4-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:50:00.652654 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:00.652634    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56409091-d2f5-43ab-9412-d95605c5e3b4-maas-api-xks-ca" pod "56409091-d2f5-43ab-9412-d95605c5e3b4" (UID: "56409091-d2f5-43ab-9412-d95605c5e3b4"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:50:00.652751 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:00.652731    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56409091-d2f5-43ab-9412-d95605c5e3b4-kube-api-access-pzf5h" pod "56409091-d2f5-43ab-9412-d95605c5e3b4" (UID: "56409091-d2f5-43ab-9412-d95605c5e3b4"). InnerVolumeSpecName "kube-api-access-pzf5h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:50:00.752413 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:00.752384    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/56409091-d2f5-43ab-9412-d95605c5e3b4-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:50:00.752413 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:00.752410    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzf5h\" (UniqueName: \"kubernetes.io/projected/56409091-d2f5-43ab-9412-d95605c5e3b4-kube-api-access-pzf5h\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:50:01.081609 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:01.081516    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b4530767-v74dj"]
Oct 30 19:50:01.090320 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:01.090291    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b4530767-v74dj"]
Oct 30 19:50:01.402376 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:01.402344    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-lmn6m"
Oct 30 19:50:01.402570 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:01.402376    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-lmn6m" event={"ID":"56409091-d2f5-43ab-9412-d95605c5e3b4","Type":"ContainerDied","Data":"61fd3091248e1ed7d7b064e0354f1c6e0b0a370ce4085666170635f6dd9d13c2"}
Oct 30 19:50:01.402570 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:01.402433    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="61fd3091248e1ed7d7b064e0354f1c6e0b0a370ce4085666170635f6dd9d13c2"
Oct 30 19:50:02.293513 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:02.293479    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c4bf9042-a613-4d6f-8d1d-ad03b6cbc152" path="/var/lib/kubelet/pods/c4bf9042-a613-4d6f-8d1d-ad03b6cbc152/volumes"
Oct 30 19:50:08.982797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:08.982700    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b9b54ffb7-4zlnt"]
Oct 30 19:50:08.983320 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:08.983043    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-6b9b54ffb7-4zlnt" podUID="af954f65-b68b-44d7-9e78-b0c249f88f38" containerName="manager" containerID="cri-o://3b79a8440912ade663a89822a3af2d5683941621deea5ccf3012ea756e04a981" gracePeriod=30
Oct 30 19:50:09.233199 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:09.233145    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b9b54ffb7-4zlnt"
Oct 30 19:50:09.340713 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:09.340677    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/af954f65-b68b-44d7-9e78-b0c249f88f38-webhook-cert\" (UniqueName: \"kubernetes.io/secret/af954f65-b68b-44d7-9e78-b0c249f88f38-webhook-cert\") pod \"af954f65-b68b-44d7-9e78-b0c249f88f38\" (UID: \"af954f65-b68b-44d7-9e78-b0c249f88f38\") "
Oct 30 19:50:09.340922 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:09.340775    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/af954f65-b68b-44d7-9e78-b0c249f88f38-metrics-certs\" (UniqueName: \"kubernetes.io/secret/af954f65-b68b-44d7-9e78-b0c249f88f38-metrics-certs\") pod \"af954f65-b68b-44d7-9e78-b0c249f88f38\" (UID: \"af954f65-b68b-44d7-9e78-b0c249f88f38\") "
Oct 30 19:50:09.340922 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:09.340824    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af954f65-b68b-44d7-9e78-b0c249f88f38-kube-api-access-drn8n\" (UniqueName: \"kubernetes.io/projected/af954f65-b68b-44d7-9e78-b0c249f88f38-kube-api-access-drn8n\") pod \"af954f65-b68b-44d7-9e78-b0c249f88f38\" (UID: \"af954f65-b68b-44d7-9e78-b0c249f88f38\") "
Oct 30 19:50:09.342846 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:09.342814    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af954f65-b68b-44d7-9e78-b0c249f88f38-webhook-cert" pod "af954f65-b68b-44d7-9e78-b0c249f88f38" (UID: "af954f65-b68b-44d7-9e78-b0c249f88f38"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:50:09.343090 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:09.343048    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af954f65-b68b-44d7-9e78-b0c249f88f38-kube-api-access-drn8n" pod "af954f65-b68b-44d7-9e78-b0c249f88f38" (UID: "af954f65-b68b-44d7-9e78-b0c249f88f38"). InnerVolumeSpecName "kube-api-access-drn8n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:50:09.343090 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:09.343052    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af954f65-b68b-44d7-9e78-b0c249f88f38-metrics-certs" pod "af954f65-b68b-44d7-9e78-b0c249f88f38" (UID: "af954f65-b68b-44d7-9e78-b0c249f88f38"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:50:09.442372 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:09.442342    5907 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/af954f65-b68b-44d7-9e78-b0c249f88f38-webhook-cert\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:50:09.442372 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:09.442370    5907 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af954f65-b68b-44d7-9e78-b0c249f88f38-metrics-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:50:09.442372 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:09.442380    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drn8n\" (UniqueName: \"kubernetes.io/projected/af954f65-b68b-44d7-9e78-b0c249f88f38-kube-api-access-drn8n\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:50:09.457105 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:09.457074    5907 generic.go:356] "Generic (PLEG): container finished" podID="af954f65-b68b-44d7-9e78-b0c249f88f38" containerID="3b79a8440912ade663a89822a3af2d5683941621deea5ccf3012ea756e04a981" exitCode=0
Oct 30 19:50:09.457250 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:09.457119    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b9b54ffb7-4zlnt"
Oct 30 19:50:09.457250 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:09.457161    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b9b54ffb7-4zlnt" event={"ID":"af954f65-b68b-44d7-9e78-b0c249f88f38","Type":"ContainerDied","Data":"3b79a8440912ade663a89822a3af2d5683941621deea5ccf3012ea756e04a981"}
Oct 30 19:50:09.457250 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:09.457215    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b9b54ffb7-4zlnt" event={"ID":"af954f65-b68b-44d7-9e78-b0c249f88f38","Type":"ContainerDied","Data":"f7821276c705c1a04dba4a26cd547b77e7998b84ffa68d967517571c390a6120"}
Oct 30 19:50:09.457250 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:09.457242    5907 scope.go:122] "RemoveContainer" containerID="3b79a8440912ade663a89822a3af2d5683941621deea5ccf3012ea756e04a981"
Oct 30 19:50:09.467564 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:09.467546    5907 scope.go:122] "RemoveContainer" containerID="3b79a8440912ade663a89822a3af2d5683941621deea5ccf3012ea756e04a981"
Oct 30 19:50:09.467830 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:50:09.467806    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b79a8440912ade663a89822a3af2d5683941621deea5ccf3012ea756e04a981\": container with ID starting with 3b79a8440912ade663a89822a3af2d5683941621deea5ccf3012ea756e04a981 not found: ID does not exist" containerID="3b79a8440912ade663a89822a3af2d5683941621deea5ccf3012ea756e04a981"
Oct 30 19:50:09.467932 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:09.467845    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b79a8440912ade663a89822a3af2d5683941621deea5ccf3012ea756e04a981"} err="failed to get container status \"3b79a8440912ade663a89822a3af2d5683941621deea5ccf3012ea756e04a981\": rpc error: code = NotFound desc = could not find container \"3b79a8440912ade663a89822a3af2d5683941621deea5ccf3012ea756e04a981\": container with ID starting with 3b79a8440912ade663a89822a3af2d5683941621deea5ccf3012ea756e04a981 not found: ID does not exist"
Oct 30 19:50:09.513232 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:09.513146    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b9b54ffb7-4zlnt"]
Oct 30 19:50:09.516892 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:09.516838    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6b9b54ffb7-4zlnt"]
Oct 30 19:50:10.286800 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:10.286765    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="af954f65-b68b-44d7-9e78-b0c249f88f38" path="/var/lib/kubelet/pods/af954f65-b68b-44d7-9e78-b0c249f88f38/volumes"
Oct 30 19:50:10.741623 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:10.741592    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6b9b54ffb7-2k2b5"]
Oct 30 19:50:10.742084 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:10.742069    5907 cpu_manager.go:414] "containerMap: removing container" podUID="af954f65-b68b-44d7-9e78-b0c249f88f38" containerName="manager"
Oct 30 19:50:10.742140 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:10.742086    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="af954f65-b68b-44d7-9e78-b0c249f88f38" containerName="manager"
Oct 30 19:50:10.742140 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:10.742115    5907 cpu_manager.go:414] "containerMap: removing container" podUID="56409091-d2f5-43ab-9412-d95605c5e3b4" containerName="revoke-keys"
Oct 30 19:50:10.742140 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:10.742122    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="56409091-d2f5-43ab-9412-d95605c5e3b4" containerName="revoke-keys"
Oct 30 19:50:10.742238 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:10.742198    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="af954f65-b68b-44d7-9e78-b0c249f88f38" containerName="manager"
Oct 30 19:50:10.742238 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:10.742209    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="56409091-d2f5-43ab-9412-d95605c5e3b4" containerName="revoke-keys"
Oct 30 19:50:10.746958 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:10.746938    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b9b54ffb7-2k2b5"
Oct 30 19:50:10.795257 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:10.795200    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 19:50:10.795426 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:10.795260    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 19:50:10.795426 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:10.795318    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 19:50:10.795426 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:10.795383    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-58qcr\""
Oct 30 19:50:10.798272 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:10.798252    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b9b54ffb7-2k2b5"]
Oct 30 19:50:10.854541 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:10.854517    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e732b2b4-b2be-4435-9d3e-5c3e744ee661-webhook-cert\") pod \"maas-controller-6b9b54ffb7-2k2b5\" (UID: \"e732b2b4-b2be-4435-9d3e-5c3e744ee661\") " pod="opendatahub/maas-controller-6b9b54ffb7-2k2b5"
Oct 30 19:50:10.854651 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:10.854556    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e732b2b4-b2be-4435-9d3e-5c3e744ee661-metrics-certs\") pod \"maas-controller-6b9b54ffb7-2k2b5\" (UID: \"e732b2b4-b2be-4435-9d3e-5c3e744ee661\") " pod="opendatahub/maas-controller-6b9b54ffb7-2k2b5"
Oct 30 19:50:10.854689 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:10.854646    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57xm5\" (UniqueName: \"kubernetes.io/projected/e732b2b4-b2be-4435-9d3e-5c3e744ee661-kube-api-access-57xm5\") pod \"maas-controller-6b9b54ffb7-2k2b5\" (UID: \"e732b2b4-b2be-4435-9d3e-5c3e744ee661\") " pod="opendatahub/maas-controller-6b9b54ffb7-2k2b5"
Oct 30 19:50:10.955410 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:10.955381    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e732b2b4-b2be-4435-9d3e-5c3e744ee661-metrics-certs\") pod \"maas-controller-6b9b54ffb7-2k2b5\" (UID: \"e732b2b4-b2be-4435-9d3e-5c3e744ee661\") " pod="opendatahub/maas-controller-6b9b54ffb7-2k2b5"
Oct 30 19:50:10.955558 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:10.955446    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57xm5\" (UniqueName: \"kubernetes.io/projected/e732b2b4-b2be-4435-9d3e-5c3e744ee661-kube-api-access-57xm5\") pod \"maas-controller-6b9b54ffb7-2k2b5\" (UID: \"e732b2b4-b2be-4435-9d3e-5c3e744ee661\") " pod="opendatahub/maas-controller-6b9b54ffb7-2k2b5"
Oct 30 19:50:10.955634 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:10.955552    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e732b2b4-b2be-4435-9d3e-5c3e744ee661-webhook-cert\") pod \"maas-controller-6b9b54ffb7-2k2b5\" (UID: \"e732b2b4-b2be-4435-9d3e-5c3e744ee661\") " pod="opendatahub/maas-controller-6b9b54ffb7-2k2b5"
Oct 30 19:50:10.957937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:10.957917    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e732b2b4-b2be-4435-9d3e-5c3e744ee661-webhook-cert\") pod \"maas-controller-6b9b54ffb7-2k2b5\" (UID: \"e732b2b4-b2be-4435-9d3e-5c3e744ee661\") " pod="opendatahub/maas-controller-6b9b54ffb7-2k2b5"
Oct 30 19:50:10.958058 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:10.957974    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e732b2b4-b2be-4435-9d3e-5c3e744ee661-metrics-certs\") pod \"maas-controller-6b9b54ffb7-2k2b5\" (UID: \"e732b2b4-b2be-4435-9d3e-5c3e744ee661\") " pod="opendatahub/maas-controller-6b9b54ffb7-2k2b5"
Oct 30 19:50:10.977154 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:10.977134    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-57xm5\" (UniqueName: \"kubernetes.io/projected/e732b2b4-b2be-4435-9d3e-5c3e744ee661-kube-api-access-57xm5\") pod \"maas-controller-6b9b54ffb7-2k2b5\" (UID: \"e732b2b4-b2be-4435-9d3e-5c3e744ee661\") " pod="opendatahub/maas-controller-6b9b54ffb7-2k2b5"
Oct 30 19:50:11.060081 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:11.060026    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b9b54ffb7-2k2b5"
Oct 30 19:50:11.360917 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:11.360890    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b9b54ffb7-2k2b5"]
Oct 30 19:50:11.362760 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:50:11.362733    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode732b2b4_b2be_4435_9d3e_5c3e744ee661.slice/crio-6762336f0b47a5a9c5f02c8dbd1acaf57b1f9004ff802b2a3c19a6fbd4ddef93 WatchSource:0}: Error finding container 6762336f0b47a5a9c5f02c8dbd1acaf57b1f9004ff802b2a3c19a6fbd4ddef93: Status 404 returned error can't find the container with id 6762336f0b47a5a9c5f02c8dbd1acaf57b1f9004ff802b2a3c19a6fbd4ddef93
Oct 30 19:50:11.472281 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:11.472248    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b9b54ffb7-2k2b5" event={"ID":"e732b2b4-b2be-4435-9d3e-5c3e744ee661","Type":"ContainerStarted","Data":"6762336f0b47a5a9c5f02c8dbd1acaf57b1f9004ff802b2a3c19a6fbd4ddef93"}
Oct 30 19:50:12.480493 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:12.480455    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b9b54ffb7-2k2b5" event={"ID":"e732b2b4-b2be-4435-9d3e-5c3e744ee661","Type":"ContainerStarted","Data":"287d89d02793d7e69ffdc0f6b8446a1f24edc7d4807e645e551278a839279d77"}
Oct 30 19:50:12.480937 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:12.480524    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6b9b54ffb7-2k2b5"
Oct 30 19:50:12.528327 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:12.528272    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-6b9b54ffb7-2k2b5" podStartSLOduration=2.144410365 podStartE2EDuration="2.528254458s" totalImagesPullingTime="383.844093ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:50:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:50:12.5262888 +0000 UTC m=+1724.369220738" watchObservedRunningTime="2026-10-30 19:50:12.528254458 +0000 UTC m=+1724.371186399"
Oct 30 19:50:17.499416 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:50:17.499381    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:50:23.489135 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:23.489106    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6b9b54ffb7-2k2b5"
Oct 30 19:50:34.222355 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:34.222319    5907 scope.go:122] "RemoveContainer" containerID="e17c9ca5a13011ad5df40c26d4ce0883bc8e2587ace8b48f1d3242757209e2e1"
Oct 30 19:50:34.232477 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:34.232457    5907 scope.go:122] "RemoveContainer" containerID="32b031228de61e18f74a60159ca0758df84028d1205b13ca836c34cf937ba214"
Oct 30 19:50:34.241101 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:34.241076    5907 scope.go:122] "RemoveContainer" containerID="a1d7aa67740480dbcc014018b3d080ea6bfc8d7a76a5fccd42c0dc64f5b9c9cb"
Oct 30 19:50:34.249248 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:34.249221    5907 scope.go:122] "RemoveContainer" containerID="6ffe00e0be15c029e66dcd280e846667d022b2fc78e9dae6b26e63773559be0d"
Oct 30 19:50:34.258223 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:34.258201    5907 scope.go:122] "RemoveContainer" containerID="920ad0de7585d8fb7f5ace99626274b09aeb1ce927a09a0285850214e7eb1740"
Oct 30 19:50:34.266370 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:34.266350    5907 scope.go:122] "RemoveContainer" containerID="6d668d845dc2e48d3b5c6cac667e97e6d87e5d3a8401fb6bf191d18835db3158"
Oct 30 19:50:34.274414 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:34.274396    5907 scope.go:122] "RemoveContainer" containerID="d45aa4bd4f858534ed86f98dc5fb1c018e4ac8f2570751314dc91252c0723279"
Oct 30 19:50:47.500881 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:50:47.500813    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:50:49.033322 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:49.033288    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-381d04c5-s8l9s"]
Oct 30 19:50:49.046680 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:49.046650    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae7b11b0-xkbwp"]
Oct 30 19:50:49.048395 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:49.048376    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-381d04c5-s8l9s"]
Oct 30 19:50:49.049934 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:49.049903    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae7b11b0-xkbwp"]
Oct 30 19:50:50.286603 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:50.286566    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3b1170ae-7298-4455-8d90-8e08d345b5a6" path="/var/lib/kubelet/pods/3b1170ae-7298-4455-8d90-8e08d345b5a6/volumes"
Oct 30 19:50:50.287034 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:50.286999    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d4e03ee0-1d1a-492a-b272-4b131d39dc4b" path="/var/lib/kubelet/pods/d4e03ee0-1d1a-492a-b272-4b131d39dc4b/volumes"
Oct 30 19:50:52.501880 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:52.501825    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:50:58.220754 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:58.220715    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4vr4k"]
Oct 30 19:50:58.225055 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:58.225034    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4vr4k"
Oct 30 19:50:58.245602 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:58.245571    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4vr4k"]
Oct 30 19:50:58.255684 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:58.255657    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:50:58.255684 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:58.255659    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:50:58.378410 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:58.378376    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4z6p\" (UniqueName: \"kubernetes.io/projected/4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0-kube-api-access-d4z6p\") pod \"maas-api-revoke-sub-c1e271a9-4vr4k\" (UID: \"4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4vr4k"
Oct 30 19:50:58.378606 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:58.378428    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c1e271a9-4vr4k\" (UID: \"4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4vr4k"
Oct 30 19:50:58.378606 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:58.378588    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c1e271a9-4vr4k\" (UID: \"4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4vr4k"
Oct 30 19:50:58.480270 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:58.480187    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4z6p\" (UniqueName: \"kubernetes.io/projected/4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0-kube-api-access-d4z6p\") pod \"maas-api-revoke-sub-c1e271a9-4vr4k\" (UID: \"4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4vr4k"
Oct 30 19:50:58.480270 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:58.480238    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c1e271a9-4vr4k\" (UID: \"4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4vr4k"
Oct 30 19:50:58.480472 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:58.480303    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c1e271a9-4vr4k\" (UID: \"4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4vr4k"
Oct 30 19:50:58.480946 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:58.480925    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c1e271a9-4vr4k\" (UID: \"4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4vr4k"
Oct 30 19:50:58.482872 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:58.482825    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c1e271a9-4vr4k\" (UID: \"4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4vr4k"
Oct 30 19:50:58.502012 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:58.501980    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4z6p\" (UniqueName: \"kubernetes.io/projected/4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0-kube-api-access-d4z6p\") pod \"maas-api-revoke-sub-c1e271a9-4vr4k\" (UID: \"4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4vr4k"
Oct 30 19:50:58.538578 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:58.538554    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4vr4k"
Oct 30 19:50:58.843374 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:58.843348    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4vr4k"]
Oct 30 19:50:58.844775 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:50:58.844751    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4de21dd4_2ce4_4ead_bfc9_d17bf7f4a7c0.slice/crio-b07d5028847f67929d1042612319e3df77bf78da088e43145eff6c5f774dc8a9 WatchSource:0}: Error finding container b07d5028847f67929d1042612319e3df77bf78da088e43145eff6c5f774dc8a9: Status 404 returned error can't find the container with id b07d5028847f67929d1042612319e3df77bf78da088e43145eff6c5f774dc8a9
Oct 30 19:50:59.797971 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:59.797936    5907 generic.go:356] "Generic (PLEG): container finished" podID="4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0" containerID="0600d617e4296fc89455c0ef1a13bb51be4788199da26b6cb1a17eaa7191495e" exitCode=0
Oct 30 19:50:59.798387 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:59.798023    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4vr4k" event={"ID":"4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0","Type":"ContainerDied","Data":"0600d617e4296fc89455c0ef1a13bb51be4788199da26b6cb1a17eaa7191495e"}
Oct 30 19:50:59.798387 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:50:59.798079    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4vr4k" event={"ID":"4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0","Type":"ContainerStarted","Data":"b07d5028847f67929d1042612319e3df77bf78da088e43145eff6c5f774dc8a9"}
Oct 30 19:51:00.943576 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:00.943548    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4vr4k"
Oct 30 19:51:01.106774 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:01.106682    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0-maas-api-openshift-ca\") pod \"4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0\" (UID: \"4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0\") "
Oct 30 19:51:01.106982 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:01.106814    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0-maas-api-xks-ca\") pod \"4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0\" (UID: \"4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0\") "
Oct 30 19:51:01.106982 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:01.106842    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0-kube-api-access-d4z6p\" (UniqueName: \"kubernetes.io/projected/4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0-kube-api-access-d4z6p\") pod \"4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0\" (UID: \"4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0\") "
Oct 30 19:51:01.107125 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:01.107098    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0-maas-api-openshift-ca" pod "4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0" (UID: "4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:51:01.107302 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:01.107281    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:51:01.109138 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:01.109118    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0-kube-api-access-d4z6p" pod "4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0" (UID: "4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0"). InnerVolumeSpecName "kube-api-access-d4z6p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:51:01.109217 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:01.109163    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0-maas-api-xks-ca" pod "4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0" (UID: "4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:51:01.208810 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:01.208776    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:51:01.208810 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:01.208805    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4z6p\" (UniqueName: \"kubernetes.io/projected/4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0-kube-api-access-d4z6p\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:51:01.812305 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:01.812274    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4vr4k"
Oct 30 19:51:01.812305 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:01.812289    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4vr4k" event={"ID":"4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0","Type":"ContainerDied","Data":"b07d5028847f67929d1042612319e3df77bf78da088e43145eff6c5f774dc8a9"}
Oct 30 19:51:01.812525 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:01.812337    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b07d5028847f67929d1042612319e3df77bf78da088e43145eff6c5f774dc8a9"
Oct 30 19:51:09.549855 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:09.549811    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:51:14.580807 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:14.580768    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:51:17.640886 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:51:17.640781    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:51:21.207285 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:21.207249    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5fsng"]
Oct 30 19:51:21.207777 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:21.207735    5907 cpu_manager.go:414] "containerMap: removing container" podUID="4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0" containerName="revoke-keys"
Oct 30 19:51:21.207777 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:21.207746    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0" containerName="revoke-keys"
Oct 30 19:51:21.207855 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:21.207830    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0" containerName="revoke-keys"
Oct 30 19:51:21.211077 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:21.211055    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5fsng"
Oct 30 19:51:21.233829 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:21.233798    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5fsng"]
Oct 30 19:51:21.252604 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:21.252566    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:51:21.252799 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:21.252781    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:51:21.361594 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:21.361560    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7eee3935-dacf-433c-b667-ef978af84873-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-5fsng\" (UID: \"7eee3935-dacf-433c-b667-ef978af84873\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5fsng"
Oct 30 19:51:21.361756 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:21.361628    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tl4wq\" (UniqueName: \"kubernetes.io/projected/7eee3935-dacf-433c-b667-ef978af84873-kube-api-access-tl4wq\") pod \"maas-api-revoke-sub-2e1f5b5f-5fsng\" (UID: \"7eee3935-dacf-433c-b667-ef978af84873\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5fsng"
Oct 30 19:51:21.361756 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:21.361694    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7eee3935-dacf-433c-b667-ef978af84873-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-5fsng\" (UID: \"7eee3935-dacf-433c-b667-ef978af84873\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5fsng"
Oct 30 19:51:21.463035 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:21.462950    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7eee3935-dacf-433c-b667-ef978af84873-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-5fsng\" (UID: \"7eee3935-dacf-433c-b667-ef978af84873\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5fsng"
Oct 30 19:51:21.463209 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:21.463043    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tl4wq\" (UniqueName: \"kubernetes.io/projected/7eee3935-dacf-433c-b667-ef978af84873-kube-api-access-tl4wq\") pod \"maas-api-revoke-sub-2e1f5b5f-5fsng\" (UID: \"7eee3935-dacf-433c-b667-ef978af84873\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5fsng"
Oct 30 19:51:21.463209 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:21.463072    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7eee3935-dacf-433c-b667-ef978af84873-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-5fsng\" (UID: \"7eee3935-dacf-433c-b667-ef978af84873\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5fsng"
Oct 30 19:51:21.463745 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:21.463726    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7eee3935-dacf-433c-b667-ef978af84873-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-5fsng\" (UID: \"7eee3935-dacf-433c-b667-ef978af84873\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5fsng"
Oct 30 19:51:21.465453 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:21.465428    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7eee3935-dacf-433c-b667-ef978af84873-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-5fsng\" (UID: \"7eee3935-dacf-433c-b667-ef978af84873\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5fsng"
Oct 30 19:51:21.494444 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:21.494423    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl4wq\" (UniqueName: \"kubernetes.io/projected/7eee3935-dacf-433c-b667-ef978af84873-kube-api-access-tl4wq\") pod \"maas-api-revoke-sub-2e1f5b5f-5fsng\" (UID: \"7eee3935-dacf-433c-b667-ef978af84873\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5fsng"
Oct 30 19:51:21.523851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:21.523826    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5fsng"
Oct 30 19:51:21.836295 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:21.836263    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5fsng"]
Oct 30 19:51:21.837316 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:51:21.837292    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eee3935_dacf_433c_b667_ef978af84873.slice/crio-f01a7daf93d75c1610a63d78e201f42e1b0029c14232b3ef0f588c29cf8c57e5 WatchSource:0}: Error finding container f01a7daf93d75c1610a63d78e201f42e1b0029c14232b3ef0f588c29cf8c57e5: Status 404 returned error can't find the container with id f01a7daf93d75c1610a63d78e201f42e1b0029c14232b3ef0f588c29cf8c57e5
Oct 30 19:51:21.971446 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:21.971404    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5fsng" event={"ID":"7eee3935-dacf-433c-b667-ef978af84873","Type":"ContainerStarted","Data":"f01a7daf93d75c1610a63d78e201f42e1b0029c14232b3ef0f588c29cf8c57e5"}
Oct 30 19:51:22.979089 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:22.979004    5907 generic.go:356] "Generic (PLEG): container finished" podID="7eee3935-dacf-433c-b667-ef978af84873" containerID="104350eb38c20f43703a9114f7e940ca96ba94ca1d3e561b4b387cd9b5aafc66" exitCode=0
Oct 30 19:51:22.979089 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:22.979052    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5fsng" event={"ID":"7eee3935-dacf-433c-b667-ef978af84873","Type":"ContainerDied","Data":"104350eb38c20f43703a9114f7e940ca96ba94ca1d3e561b4b387cd9b5aafc66"}
Oct 30 19:51:24.028681 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:24.028641    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-57e671e1-fjvwq"]
Oct 30 19:51:24.043185 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:24.043146    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7a47e70e-xrzvf"]
Oct 30 19:51:24.045157 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:24.045132    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-57e671e1-fjvwq"]
Oct 30 19:51:24.047617 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:24.047593    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7a47e70e-xrzvf"]
Oct 30 19:51:24.125941 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:24.125916    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5fsng"
Oct 30 19:51:24.286300 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:24.286221    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9dd73dbe-af1c-4d56-82e3-228644efda7a" path="/var/lib/kubelet/pods/9dd73dbe-af1c-4d56-82e3-228644efda7a/volumes"
Oct 30 19:51:24.286646 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:24.286633    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b9d6296e-59ef-4853-8470-d851752d8694" path="/var/lib/kubelet/pods/b9d6296e-59ef-4853-8470-d851752d8694/volumes"
Oct 30 19:51:24.290768 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:24.290739    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7eee3935-dacf-433c-b667-ef978af84873-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7eee3935-dacf-433c-b667-ef978af84873-maas-api-xks-ca\") pod \"7eee3935-dacf-433c-b667-ef978af84873\" (UID: \"7eee3935-dacf-433c-b667-ef978af84873\") "
Oct 30 19:51:24.290956 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:24.290938    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7eee3935-dacf-433c-b667-ef978af84873-kube-api-access-tl4wq\" (UniqueName: \"kubernetes.io/projected/7eee3935-dacf-433c-b667-ef978af84873-kube-api-access-tl4wq\") pod \"7eee3935-dacf-433c-b667-ef978af84873\" (UID: \"7eee3935-dacf-433c-b667-ef978af84873\") "
Oct 30 19:51:24.291013 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:24.290980    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7eee3935-dacf-433c-b667-ef978af84873-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7eee3935-dacf-433c-b667-ef978af84873-maas-api-openshift-ca\") pod \"7eee3935-dacf-433c-b667-ef978af84873\" (UID: \"7eee3935-dacf-433c-b667-ef978af84873\") "
Oct 30 19:51:24.291358 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:24.291338    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7eee3935-dacf-433c-b667-ef978af84873-maas-api-openshift-ca" pod "7eee3935-dacf-433c-b667-ef978af84873" (UID: "7eee3935-dacf-433c-b667-ef978af84873"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:51:24.292983 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:24.292963    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7eee3935-dacf-433c-b667-ef978af84873-maas-api-xks-ca" pod "7eee3935-dacf-433c-b667-ef978af84873" (UID: "7eee3935-dacf-433c-b667-ef978af84873"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:51:24.293145 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:24.293123    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7eee3935-dacf-433c-b667-ef978af84873-kube-api-access-tl4wq" pod "7eee3935-dacf-433c-b667-ef978af84873" (UID: "7eee3935-dacf-433c-b667-ef978af84873"). InnerVolumeSpecName "kube-api-access-tl4wq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:51:24.392345 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:24.392302    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7eee3935-dacf-433c-b667-ef978af84873-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:51:24.392345 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:24.392342    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tl4wq\" (UniqueName: \"kubernetes.io/projected/7eee3935-dacf-433c-b667-ef978af84873-kube-api-access-tl4wq\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:51:24.392581 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:24.392358    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7eee3935-dacf-433c-b667-ef978af84873-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:51:24.992024 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:24.991989    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5fsng" event={"ID":"7eee3935-dacf-433c-b667-ef978af84873","Type":"ContainerDied","Data":"f01a7daf93d75c1610a63d78e201f42e1b0029c14232b3ef0f588c29cf8c57e5"}
Oct 30 19:51:24.992222 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:24.992030    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5fsng"
Oct 30 19:51:24.992222 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:24.992039    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f01a7daf93d75c1610a63d78e201f42e1b0029c14232b3ef0f588c29cf8c57e5"
Oct 30 19:51:32.532204 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:32.532171    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-zhs4l"]
Oct 30 19:51:32.532653 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:32.532640    5907 cpu_manager.go:414] "containerMap: removing container" podUID="7eee3935-dacf-433c-b667-ef978af84873" containerName="revoke-keys"
Oct 30 19:51:32.532695 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:32.532654    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7eee3935-dacf-433c-b667-ef978af84873" containerName="revoke-keys"
Oct 30 19:51:32.532736 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:32.532726    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="7eee3935-dacf-433c-b667-ef978af84873" containerName="revoke-keys"
Oct 30 19:51:32.536136 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:32.536115    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-zhs4l"
Oct 30 19:51:32.555013 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:32.554976    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:51:32.568540 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:32.568509    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:51:32.572442 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:32.572411    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-zhs4l"]
Oct 30 19:51:32.675312 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:32.675275    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/31af6090-882d-4de8-9a4d-4aae8239e8e5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec843018-zhs4l\" (UID: \"31af6090-882d-4de8-9a4d-4aae8239e8e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-zhs4l"
Oct 30 19:51:32.675312 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:32.675325    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss96n\" (UniqueName: \"kubernetes.io/projected/31af6090-882d-4de8-9a4d-4aae8239e8e5-kube-api-access-ss96n\") pod \"maas-api-revoke-sub-ec843018-zhs4l\" (UID: \"31af6090-882d-4de8-9a4d-4aae8239e8e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-zhs4l"
Oct 30 19:51:32.675582 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:32.675493    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/31af6090-882d-4de8-9a4d-4aae8239e8e5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec843018-zhs4l\" (UID: \"31af6090-882d-4de8-9a4d-4aae8239e8e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-zhs4l"
Oct 30 19:51:32.776281 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:32.776245    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/31af6090-882d-4de8-9a4d-4aae8239e8e5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec843018-zhs4l\" (UID: \"31af6090-882d-4de8-9a4d-4aae8239e8e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-zhs4l"
Oct 30 19:51:32.776281 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:32.776284    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss96n\" (UniqueName: \"kubernetes.io/projected/31af6090-882d-4de8-9a4d-4aae8239e8e5-kube-api-access-ss96n\") pod \"maas-api-revoke-sub-ec843018-zhs4l\" (UID: \"31af6090-882d-4de8-9a4d-4aae8239e8e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-zhs4l"
Oct 30 19:51:32.776578 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:32.776357    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/31af6090-882d-4de8-9a4d-4aae8239e8e5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec843018-zhs4l\" (UID: \"31af6090-882d-4de8-9a4d-4aae8239e8e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-zhs4l"
Oct 30 19:51:32.777040 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:32.777017    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/31af6090-882d-4de8-9a4d-4aae8239e8e5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec843018-zhs4l\" (UID: \"31af6090-882d-4de8-9a4d-4aae8239e8e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-zhs4l"
Oct 30 19:51:32.778931 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:32.778911    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/31af6090-882d-4de8-9a4d-4aae8239e8e5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec843018-zhs4l\" (UID: \"31af6090-882d-4de8-9a4d-4aae8239e8e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-zhs4l"
Oct 30 19:51:32.803620 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:32.803550    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss96n\" (UniqueName: \"kubernetes.io/projected/31af6090-882d-4de8-9a4d-4aae8239e8e5-kube-api-access-ss96n\") pod \"maas-api-revoke-sub-ec843018-zhs4l\" (UID: \"31af6090-882d-4de8-9a4d-4aae8239e8e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-zhs4l"
Oct 30 19:51:32.850692 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:32.850660    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-zhs4l"
Oct 30 19:51:33.159098 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:33.159071    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-zhs4l"]
Oct 30 19:51:33.161422 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:51:33.161394    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31af6090_882d_4de8_9a4d_4aae8239e8e5.slice/crio-ea3d33037cbeb59950f5352d1e154e509fa3959f9a91bf10b06b3023ddb7d5ca WatchSource:0}: Error finding container ea3d33037cbeb59950f5352d1e154e509fa3959f9a91bf10b06b3023ddb7d5ca: Status 404 returned error can't find the container with id ea3d33037cbeb59950f5352d1e154e509fa3959f9a91bf10b06b3023ddb7d5ca
Oct 30 19:51:34.057110 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:34.057074    5907 generic.go:356] "Generic (PLEG): container finished" podID="31af6090-882d-4de8-9a4d-4aae8239e8e5" containerID="4e4b84711c327742a55bfb4cecccfeb517978e1d2a9892f2cabccad662cbeb04" exitCode=0
Oct 30 19:51:34.057529 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:34.057108    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-zhs4l" event={"ID":"31af6090-882d-4de8-9a4d-4aae8239e8e5","Type":"ContainerDied","Data":"4e4b84711c327742a55bfb4cecccfeb517978e1d2a9892f2cabccad662cbeb04"}
Oct 30 19:51:34.057529 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:34.057190    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-zhs4l" event={"ID":"31af6090-882d-4de8-9a4d-4aae8239e8e5","Type":"ContainerStarted","Data":"ea3d33037cbeb59950f5352d1e154e509fa3959f9a91bf10b06b3023ddb7d5ca"}
Oct 30 19:51:34.329129 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:34.329052    5907 scope.go:122] "RemoveContainer" containerID="aff22d4616dc59538c5919e28607117b6aaef0f6f3fee9a5b728a9dad011a7ad"
Oct 30 19:51:34.338946 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:34.338928    5907 scope.go:122] "RemoveContainer" containerID="8630df26dca3f1d1b902c97f9fa09670c86f5763c8d2e92d57b820f5124eb4b2"
Oct 30 19:51:34.348931 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:34.348906    5907 scope.go:122] "RemoveContainer" containerID="571aa497c82e6d6a784874f21a87b2a1a7fe1d2502b46c0f3fb796790970020a"
Oct 30 19:51:34.660064 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:34.660032    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:51:35.219455 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:35.219422    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-zhs4l"
Oct 30 19:51:35.301164 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:35.301133    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/31af6090-882d-4de8-9a4d-4aae8239e8e5-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/31af6090-882d-4de8-9a4d-4aae8239e8e5-maas-api-openshift-ca\") pod \"31af6090-882d-4de8-9a4d-4aae8239e8e5\" (UID: \"31af6090-882d-4de8-9a4d-4aae8239e8e5\") "
Oct 30 19:51:35.301366 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:35.301210    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/31af6090-882d-4de8-9a4d-4aae8239e8e5-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/31af6090-882d-4de8-9a4d-4aae8239e8e5-maas-api-xks-ca\") pod \"31af6090-882d-4de8-9a4d-4aae8239e8e5\" (UID: \"31af6090-882d-4de8-9a4d-4aae8239e8e5\") "
Oct 30 19:51:35.301366 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:35.301257    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/31af6090-882d-4de8-9a4d-4aae8239e8e5-kube-api-access-ss96n\" (UniqueName: \"kubernetes.io/projected/31af6090-882d-4de8-9a4d-4aae8239e8e5-kube-api-access-ss96n\") pod \"31af6090-882d-4de8-9a4d-4aae8239e8e5\" (UID: \"31af6090-882d-4de8-9a4d-4aae8239e8e5\") "
Oct 30 19:51:35.301584 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:35.301558    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31af6090-882d-4de8-9a4d-4aae8239e8e5-maas-api-openshift-ca" pod "31af6090-882d-4de8-9a4d-4aae8239e8e5" (UID: "31af6090-882d-4de8-9a4d-4aae8239e8e5"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:51:35.303560 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:35.303530    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31af6090-882d-4de8-9a4d-4aae8239e8e5-kube-api-access-ss96n" pod "31af6090-882d-4de8-9a4d-4aae8239e8e5" (UID: "31af6090-882d-4de8-9a4d-4aae8239e8e5"). InnerVolumeSpecName "kube-api-access-ss96n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:51:35.303560 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:35.303530    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31af6090-882d-4de8-9a4d-4aae8239e8e5-maas-api-xks-ca" pod "31af6090-882d-4de8-9a4d-4aae8239e8e5" (UID: "31af6090-882d-4de8-9a4d-4aae8239e8e5"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:51:35.402376 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:35.402339    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ss96n\" (UniqueName: \"kubernetes.io/projected/31af6090-882d-4de8-9a4d-4aae8239e8e5-kube-api-access-ss96n\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:51:35.402376 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:35.402373    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/31af6090-882d-4de8-9a4d-4aae8239e8e5-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:51:35.402376 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:35.402383    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/31af6090-882d-4de8-9a4d-4aae8239e8e5-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:51:36.072975 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:36.072940    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-zhs4l" event={"ID":"31af6090-882d-4de8-9a4d-4aae8239e8e5","Type":"ContainerDied","Data":"ea3d33037cbeb59950f5352d1e154e509fa3959f9a91bf10b06b3023ddb7d5ca"}
Oct 30 19:51:36.072975 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:36.072977    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-zhs4l"
Oct 30 19:51:36.073204 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:36.072992    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ea3d33037cbeb59950f5352d1e154e509fa3959f9a91bf10b06b3023ddb7d5ca"
Oct 30 19:51:47.502362 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:51:47.502272    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:51:48.370752 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:48.370717    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:51:50.851888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:50.850190    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-9tdql container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:51:50.851888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:50.850279    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9tdql" podUID="8db50859-98fc-49aa-806b-4fbed56b1d00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:51:52.030081 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:52.030039    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b8eda0f9-59xt4"]
Oct 30 19:51:52.043759 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:52.043733    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-967ecdfb-8qbwl"]
Oct 30 19:51:52.045650 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:52.045628    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b8eda0f9-59xt4"]
Oct 30 19:51:52.048340 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:52.048317    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-967ecdfb-8qbwl"]
Oct 30 19:51:52.286544 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:52.286466    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9eb15025-db65-4deb-8a88-d6302ab53996" path="/var/lib/kubelet/pods/9eb15025-db65-4deb-8a88-d6302ab53996/volumes"
Oct 30 19:51:52.286923 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:51:52.286905    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e3eabdcb-14c1-446c-88d9-261df5e254cb" path="/var/lib/kubelet/pods/e3eabdcb-14c1-446c-88d9-261df5e254cb/volumes"
Oct 30 19:52:00.297289 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:00.297248    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-drrmq"]
Oct 30 19:52:00.297833 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:00.297768    5907 cpu_manager.go:414] "containerMap: removing container" podUID="31af6090-882d-4de8-9a4d-4aae8239e8e5" containerName="revoke-keys"
Oct 30 19:52:00.297833 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:00.297781    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="31af6090-882d-4de8-9a4d-4aae8239e8e5" containerName="revoke-keys"
Oct 30 19:52:00.298101 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:00.297877    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="31af6090-882d-4de8-9a4d-4aae8239e8e5" containerName="revoke-keys"
Oct 30 19:52:00.302186 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:00.302171    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-drrmq"
Oct 30 19:52:00.318581 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:00.318555    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:52:00.332087 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:00.332063    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:52:00.334229 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:00.334204    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-drrmq"]
Oct 30 19:52:00.397912 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:00.397851    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xv9m\" (UniqueName: \"kubernetes.io/projected/de2eb2ff-8abf-479e-9297-92174f893b1a-kube-api-access-2xv9m\") pod \"maas-api-revoke-sub-046e1343-drrmq\" (UID: \"de2eb2ff-8abf-479e-9297-92174f893b1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-drrmq"
Oct 30 19:52:00.398106 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:00.397922    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/de2eb2ff-8abf-479e-9297-92174f893b1a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-046e1343-drrmq\" (UID: \"de2eb2ff-8abf-479e-9297-92174f893b1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-drrmq"
Oct 30 19:52:00.398106 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:00.398045    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/de2eb2ff-8abf-479e-9297-92174f893b1a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-046e1343-drrmq\" (UID: \"de2eb2ff-8abf-479e-9297-92174f893b1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-drrmq"
Oct 30 19:52:00.499227 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:00.499186    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/de2eb2ff-8abf-479e-9297-92174f893b1a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-046e1343-drrmq\" (UID: \"de2eb2ff-8abf-479e-9297-92174f893b1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-drrmq"
Oct 30 19:52:00.499386 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:00.499334    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xv9m\" (UniqueName: \"kubernetes.io/projected/de2eb2ff-8abf-479e-9297-92174f893b1a-kube-api-access-2xv9m\") pod \"maas-api-revoke-sub-046e1343-drrmq\" (UID: \"de2eb2ff-8abf-479e-9297-92174f893b1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-drrmq"
Oct 30 19:52:00.499386 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:00.499366    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/de2eb2ff-8abf-479e-9297-92174f893b1a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-046e1343-drrmq\" (UID: \"de2eb2ff-8abf-479e-9297-92174f893b1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-drrmq"
Oct 30 19:52:00.499976 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:00.499953    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/de2eb2ff-8abf-479e-9297-92174f893b1a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-046e1343-drrmq\" (UID: \"de2eb2ff-8abf-479e-9297-92174f893b1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-drrmq"
Oct 30 19:52:00.501834 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:00.501817    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/de2eb2ff-8abf-479e-9297-92174f893b1a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-046e1343-drrmq\" (UID: \"de2eb2ff-8abf-479e-9297-92174f893b1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-drrmq"
Oct 30 19:52:00.521377 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:00.521343    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xv9m\" (UniqueName: \"kubernetes.io/projected/de2eb2ff-8abf-479e-9297-92174f893b1a-kube-api-access-2xv9m\") pod \"maas-api-revoke-sub-046e1343-drrmq\" (UID: \"de2eb2ff-8abf-479e-9297-92174f893b1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-drrmq"
Oct 30 19:52:00.614985 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:00.614961    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-drrmq"
Oct 30 19:52:00.916018 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:00.915994    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-drrmq"]
Oct 30 19:52:00.918070 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:52:00.918038    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde2eb2ff_8abf_479e_9297_92174f893b1a.slice/crio-de7b497c26ac88d8aae1f7039e8d4463e9d4a8320c186b64b74be620143b21ae WatchSource:0}: Error finding container de7b497c26ac88d8aae1f7039e8d4463e9d4a8320c186b64b74be620143b21ae: Status 404 returned error can't find the container with id de7b497c26ac88d8aae1f7039e8d4463e9d4a8320c186b64b74be620143b21ae
Oct 30 19:52:01.253406 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:01.253368    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-drrmq" event={"ID":"de2eb2ff-8abf-479e-9297-92174f893b1a","Type":"ContainerStarted","Data":"c9729ee73cf998d8b9a15c7336d7dcd2162bee5e58c9eb05b0121f7da2c74e54"}
Oct 30 19:52:01.253558 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:01.253435    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-drrmq" event={"ID":"de2eb2ff-8abf-479e-9297-92174f893b1a","Type":"ContainerStarted","Data":"de7b497c26ac88d8aae1f7039e8d4463e9d4a8320c186b64b74be620143b21ae"}
Oct 30 19:52:01.296413 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:01.296368    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-drrmq" podStartSLOduration=1.296354339 podStartE2EDuration="1.296354339s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:52:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:52:01.294731527 +0000 UTC m=+1833.137663466" watchObservedRunningTime="2026-10-30 19:52:01.296354339 +0000 UTC m=+1833.139286272"
Oct 30 19:52:02.260805 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:02.260770    5907 generic.go:356] "Generic (PLEG): container finished" podID="de2eb2ff-8abf-479e-9297-92174f893b1a" containerID="c9729ee73cf998d8b9a15c7336d7dcd2162bee5e58c9eb05b0121f7da2c74e54" exitCode=0
Oct 30 19:52:02.261239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:02.260883    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-drrmq" event={"ID":"de2eb2ff-8abf-479e-9297-92174f893b1a","Type":"ContainerDied","Data":"c9729ee73cf998d8b9a15c7336d7dcd2162bee5e58c9eb05b0121f7da2c74e54"}
Oct 30 19:52:03.409806 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:03.409782    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-drrmq"
Oct 30 19:52:03.531230 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:03.531145    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/de2eb2ff-8abf-479e-9297-92174f893b1a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/de2eb2ff-8abf-479e-9297-92174f893b1a-maas-api-openshift-ca\") pod \"de2eb2ff-8abf-479e-9297-92174f893b1a\" (UID: \"de2eb2ff-8abf-479e-9297-92174f893b1a\") "
Oct 30 19:52:03.531421 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:03.531256    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/de2eb2ff-8abf-479e-9297-92174f893b1a-kube-api-access-2xv9m\" (UniqueName: \"kubernetes.io/projected/de2eb2ff-8abf-479e-9297-92174f893b1a-kube-api-access-2xv9m\") pod \"de2eb2ff-8abf-479e-9297-92174f893b1a\" (UID: \"de2eb2ff-8abf-479e-9297-92174f893b1a\") "
Oct 30 19:52:03.531421 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:03.531302    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/de2eb2ff-8abf-479e-9297-92174f893b1a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/de2eb2ff-8abf-479e-9297-92174f893b1a-maas-api-xks-ca\") pod \"de2eb2ff-8abf-479e-9297-92174f893b1a\" (UID: \"de2eb2ff-8abf-479e-9297-92174f893b1a\") "
Oct 30 19:52:03.531552 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:03.531530    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de2eb2ff-8abf-479e-9297-92174f893b1a-maas-api-openshift-ca" pod "de2eb2ff-8abf-479e-9297-92174f893b1a" (UID: "de2eb2ff-8abf-479e-9297-92174f893b1a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:52:03.531728 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:03.531710    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/de2eb2ff-8abf-479e-9297-92174f893b1a-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:52:03.533562 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:03.533538    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de2eb2ff-8abf-479e-9297-92174f893b1a-maas-api-xks-ca" pod "de2eb2ff-8abf-479e-9297-92174f893b1a" (UID: "de2eb2ff-8abf-479e-9297-92174f893b1a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:52:03.533642 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:03.533599    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de2eb2ff-8abf-479e-9297-92174f893b1a-kube-api-access-2xv9m" pod "de2eb2ff-8abf-479e-9297-92174f893b1a" (UID: "de2eb2ff-8abf-479e-9297-92174f893b1a"). InnerVolumeSpecName "kube-api-access-2xv9m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:52:03.632742 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:03.632707    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2xv9m\" (UniqueName: \"kubernetes.io/projected/de2eb2ff-8abf-479e-9297-92174f893b1a-kube-api-access-2xv9m\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:52:03.632742 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:03.632732    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/de2eb2ff-8abf-479e-9297-92174f893b1a-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:52:04.273443 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:04.273404    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-drrmq"
Oct 30 19:52:04.273663 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:04.273439    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-drrmq" event={"ID":"de2eb2ff-8abf-479e-9297-92174f893b1a","Type":"ContainerDied","Data":"de7b497c26ac88d8aae1f7039e8d4463e9d4a8320c186b64b74be620143b21ae"}
Oct 30 19:52:04.273663 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:04.273543    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="de7b497c26ac88d8aae1f7039e8d4463e9d4a8320c186b64b74be620143b21ae"
Oct 30 19:52:10.632740 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:10.632658    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-7r7bc"]
Oct 30 19:52:10.633361 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:10.633339    5907 cpu_manager.go:414] "containerMap: removing container" podUID="de2eb2ff-8abf-479e-9297-92174f893b1a" containerName="revoke-keys"
Oct 30 19:52:10.633457 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:10.633363    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="de2eb2ff-8abf-479e-9297-92174f893b1a" containerName="revoke-keys"
Oct 30 19:52:10.633528 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:10.633494    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="de2eb2ff-8abf-479e-9297-92174f893b1a" containerName="revoke-keys"
Oct 30 19:52:10.637163 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:10.637139    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-7r7bc"
Oct 30 19:52:10.659476 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:10.659448    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-7r7bc"]
Oct 30 19:52:10.669111 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:10.669082    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:52:10.669275 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:10.669118    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:52:10.805184 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:10.805151    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b8916631-dd19-4062-b5e6-1d68976378dd-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d3fba056-7r7bc\" (UID: \"b8916631-dd19-4062-b5e6-1d68976378dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-7r7bc"
Oct 30 19:52:10.805355 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:10.805234    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b8916631-dd19-4062-b5e6-1d68976378dd-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d3fba056-7r7bc\" (UID: \"b8916631-dd19-4062-b5e6-1d68976378dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-7r7bc"
Oct 30 19:52:10.805355 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:10.805333    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5jsf\" (UniqueName: \"kubernetes.io/projected/b8916631-dd19-4062-b5e6-1d68976378dd-kube-api-access-c5jsf\") pod \"maas-api-revoke-sub-d3fba056-7r7bc\" (UID: \"b8916631-dd19-4062-b5e6-1d68976378dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-7r7bc"
Oct 30 19:52:10.906726 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:10.906655    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b8916631-dd19-4062-b5e6-1d68976378dd-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d3fba056-7r7bc\" (UID: \"b8916631-dd19-4062-b5e6-1d68976378dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-7r7bc"
Oct 30 19:52:10.906726 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:10.906712    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5jsf\" (UniqueName: \"kubernetes.io/projected/b8916631-dd19-4062-b5e6-1d68976378dd-kube-api-access-c5jsf\") pod \"maas-api-revoke-sub-d3fba056-7r7bc\" (UID: \"b8916631-dd19-4062-b5e6-1d68976378dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-7r7bc"
Oct 30 19:52:10.906933 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:10.906763    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b8916631-dd19-4062-b5e6-1d68976378dd-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d3fba056-7r7bc\" (UID: \"b8916631-dd19-4062-b5e6-1d68976378dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-7r7bc"
Oct 30 19:52:10.907289 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:10.907269    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b8916631-dd19-4062-b5e6-1d68976378dd-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d3fba056-7r7bc\" (UID: \"b8916631-dd19-4062-b5e6-1d68976378dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-7r7bc"
Oct 30 19:52:10.909283 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:10.909258    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b8916631-dd19-4062-b5e6-1d68976378dd-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d3fba056-7r7bc\" (UID: \"b8916631-dd19-4062-b5e6-1d68976378dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-7r7bc"
Oct 30 19:52:10.928499 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:10.928474    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5jsf\" (UniqueName: \"kubernetes.io/projected/b8916631-dd19-4062-b5e6-1d68976378dd-kube-api-access-c5jsf\") pod \"maas-api-revoke-sub-d3fba056-7r7bc\" (UID: \"b8916631-dd19-4062-b5e6-1d68976378dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-7r7bc"
Oct 30 19:52:10.955170 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:10.955148    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-7r7bc"
Oct 30 19:52:11.261987 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:11.261964    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-7r7bc"]
Oct 30 19:52:11.264069 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:52:11.264027    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8916631_dd19_4062_b5e6_1d68976378dd.slice/crio-ac9ec67bf7f53607062ba4afadd58b22bd847a652e9da04263d592aa2c52b0c3 WatchSource:0}: Error finding container ac9ec67bf7f53607062ba4afadd58b22bd847a652e9da04263d592aa2c52b0c3: Status 404 returned error can't find the container with id ac9ec67bf7f53607062ba4afadd58b22bd847a652e9da04263d592aa2c52b0c3
Oct 30 19:52:11.316628 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:11.316600    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-7r7bc" event={"ID":"b8916631-dd19-4062-b5e6-1d68976378dd","Type":"ContainerStarted","Data":"ac9ec67bf7f53607062ba4afadd58b22bd847a652e9da04263d592aa2c52b0c3"}
Oct 30 19:52:11.665024 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:11.664995    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:52:12.327450 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:12.327253    5907 generic.go:356] "Generic (PLEG): container finished" podID="b8916631-dd19-4062-b5e6-1d68976378dd" containerID="2c07c8eb538c4c29cab8f8aa764320b532b90b18627d373fa820de1c68e4b589" exitCode=0
Oct 30 19:52:12.327450 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:12.327344    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-7r7bc" event={"ID":"b8916631-dd19-4062-b5e6-1d68976378dd","Type":"ContainerDied","Data":"2c07c8eb538c4c29cab8f8aa764320b532b90b18627d373fa820de1c68e4b589"}
Oct 30 19:52:14.264962 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:14.264929    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-7r7bc"
Oct 30 19:52:14.424385 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:14.424289    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b8916631-dd19-4062-b5e6-1d68976378dd-kube-api-access-c5jsf\" (UniqueName: \"kubernetes.io/projected/b8916631-dd19-4062-b5e6-1d68976378dd-kube-api-access-c5jsf\") pod \"b8916631-dd19-4062-b5e6-1d68976378dd\" (UID: \"b8916631-dd19-4062-b5e6-1d68976378dd\") "
Oct 30 19:52:14.425120 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:14.425084    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b8916631-dd19-4062-b5e6-1d68976378dd-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b8916631-dd19-4062-b5e6-1d68976378dd-maas-api-openshift-ca\") pod \"b8916631-dd19-4062-b5e6-1d68976378dd\" (UID: \"b8916631-dd19-4062-b5e6-1d68976378dd\") "
Oct 30 19:52:14.425765 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:14.425634    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8916631-dd19-4062-b5e6-1d68976378dd-maas-api-openshift-ca" pod "b8916631-dd19-4062-b5e6-1d68976378dd" (UID: "b8916631-dd19-4062-b5e6-1d68976378dd"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:52:14.426003 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:14.425985    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b8916631-dd19-4062-b5e6-1d68976378dd-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b8916631-dd19-4062-b5e6-1d68976378dd-maas-api-xks-ca\") pod \"b8916631-dd19-4062-b5e6-1d68976378dd\" (UID: \"b8916631-dd19-4062-b5e6-1d68976378dd\") "
Oct 30 19:52:14.426809 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:14.426788    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b8916631-dd19-4062-b5e6-1d68976378dd-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:52:14.430484 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:14.430456    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-7r7bc" event={"ID":"b8916631-dd19-4062-b5e6-1d68976378dd","Type":"ContainerDied","Data":"ac9ec67bf7f53607062ba4afadd58b22bd847a652e9da04263d592aa2c52b0c3"}
Oct 30 19:52:14.430664 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:14.430647    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ac9ec67bf7f53607062ba4afadd58b22bd847a652e9da04263d592aa2c52b0c3"
Oct 30 19:52:14.430845 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:14.430830    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-7r7bc"
Oct 30 19:52:14.448946 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:14.448844    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8916631-dd19-4062-b5e6-1d68976378dd-kube-api-access-c5jsf" pod "b8916631-dd19-4062-b5e6-1d68976378dd" (UID: "b8916631-dd19-4062-b5e6-1d68976378dd"). InnerVolumeSpecName "kube-api-access-c5jsf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:52:14.455481 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:14.455437    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8916631-dd19-4062-b5e6-1d68976378dd-maas-api-xks-ca" pod "b8916631-dd19-4062-b5e6-1d68976378dd" (UID: "b8916631-dd19-4062-b5e6-1d68976378dd"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:52:14.532654 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:14.532581    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c5jsf\" (UniqueName: \"kubernetes.io/projected/b8916631-dd19-4062-b5e6-1d68976378dd-kube-api-access-c5jsf\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:52:14.532654 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:14.532620    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b8916631-dd19-4062-b5e6-1d68976378dd-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:52:17.503053 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:52:17.503013    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:52:22.000883 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:22.000834    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:52:26.962221 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:26.962186    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:52:32.273528 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:32.273485    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-tg6wg"]
Oct 30 19:52:32.274075 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:32.274053    5907 cpu_manager.go:414] "containerMap: removing container" podUID="b8916631-dd19-4062-b5e6-1d68976378dd" containerName="revoke-keys"
Oct 30 19:52:32.274075 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:32.274071    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b8916631-dd19-4062-b5e6-1d68976378dd" containerName="revoke-keys"
Oct 30 19:52:32.274207 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:32.274193    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="b8916631-dd19-4062-b5e6-1d68976378dd" containerName="revoke-keys"
Oct 30 19:52:32.277718 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:32.277700    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-tg6wg"
Oct 30 19:52:32.297569 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:32.297539    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-tg6wg"]
Oct 30 19:52:32.307925 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:32.307899    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:52:32.308098 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:32.307940    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:52:32.332177 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:32.332141    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0db10165-3412-4936-86b2-a7ec5e258915-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ae594979-tg6wg\" (UID: \"0db10165-3412-4936-86b2-a7ec5e258915\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-tg6wg"
Oct 30 19:52:32.332504 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:32.332483    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7c8x\" (UniqueName: \"kubernetes.io/projected/0db10165-3412-4936-86b2-a7ec5e258915-kube-api-access-k7c8x\") pod \"maas-api-revoke-sub-ae594979-tg6wg\" (UID: \"0db10165-3412-4936-86b2-a7ec5e258915\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-tg6wg"
Oct 30 19:52:32.332647 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:32.332633    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0db10165-3412-4936-86b2-a7ec5e258915-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ae594979-tg6wg\" (UID: \"0db10165-3412-4936-86b2-a7ec5e258915\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-tg6wg"
Oct 30 19:52:32.433735 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:32.433697    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7c8x\" (UniqueName: \"kubernetes.io/projected/0db10165-3412-4936-86b2-a7ec5e258915-kube-api-access-k7c8x\") pod \"maas-api-revoke-sub-ae594979-tg6wg\" (UID: \"0db10165-3412-4936-86b2-a7ec5e258915\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-tg6wg"
Oct 30 19:52:32.433735 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:32.433741    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0db10165-3412-4936-86b2-a7ec5e258915-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ae594979-tg6wg\" (UID: \"0db10165-3412-4936-86b2-a7ec5e258915\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-tg6wg"
Oct 30 19:52:32.433988 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:32.433819    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0db10165-3412-4936-86b2-a7ec5e258915-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ae594979-tg6wg\" (UID: \"0db10165-3412-4936-86b2-a7ec5e258915\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-tg6wg"
Oct 30 19:52:32.434467 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:32.434435    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0db10165-3412-4936-86b2-a7ec5e258915-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ae594979-tg6wg\" (UID: \"0db10165-3412-4936-86b2-a7ec5e258915\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-tg6wg"
Oct 30 19:52:32.436310 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:32.436284    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0db10165-3412-4936-86b2-a7ec5e258915-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ae594979-tg6wg\" (UID: \"0db10165-3412-4936-86b2-a7ec5e258915\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-tg6wg"
Oct 30 19:52:32.454534 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:32.454512    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7c8x\" (UniqueName: \"kubernetes.io/projected/0db10165-3412-4936-86b2-a7ec5e258915-kube-api-access-k7c8x\") pod \"maas-api-revoke-sub-ae594979-tg6wg\" (UID: \"0db10165-3412-4936-86b2-a7ec5e258915\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-tg6wg"
Oct 30 19:52:32.592291 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:32.592218    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-tg6wg"
Oct 30 19:52:32.891927 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:32.891904    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-tg6wg"]
Oct 30 19:52:32.894370 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:52:32.894343    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0db10165_3412_4936_86b2_a7ec5e258915.slice/crio-1a19677abfe5eaa2913da047f165593316078ae7acaf01bf1a23d7d9d0443b1a WatchSource:0}: Error finding container 1a19677abfe5eaa2913da047f165593316078ae7acaf01bf1a23d7d9d0443b1a: Status 404 returned error can't find the container with id 1a19677abfe5eaa2913da047f165593316078ae7acaf01bf1a23d7d9d0443b1a
Oct 30 19:52:33.572247 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:33.572156    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-tg6wg" event={"ID":"0db10165-3412-4936-86b2-a7ec5e258915","Type":"ContainerStarted","Data":"1c46fad432ed6a4989fca4ab03e3db0cf49ab26c9ffbedb5db1c11c5717f3fa9"}
Oct 30 19:52:33.572247 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:33.572225    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-tg6wg" event={"ID":"0db10165-3412-4936-86b2-a7ec5e258915","Type":"ContainerStarted","Data":"1a19677abfe5eaa2913da047f165593316078ae7acaf01bf1a23d7d9d0443b1a"}
Oct 30 19:52:33.613659 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:33.613612    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-tg6wg" podStartSLOduration=1.613598681 podStartE2EDuration="1.613598681s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:52:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:52:33.612228755 +0000 UTC m=+1865.455160676" watchObservedRunningTime="2026-10-30 19:52:33.613598681 +0000 UTC m=+1865.456530632"
Oct 30 19:52:34.380596 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:34.380561    5907 scope.go:122] "RemoveContainer" containerID="7f8e72dbac689c04892c385ba54c4e3e7e53c52d28d39d917d6fd90ef0ae0452"
Oct 30 19:52:34.390067 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:34.390049    5907 scope.go:122] "RemoveContainer" containerID="a757bde9aef867d49a4b10aafa37d80eb77f56c84dc1b6a47ecfe87f7df9a7fb"
Oct 30 19:52:34.401174 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:34.401156    5907 scope.go:122] "RemoveContainer" containerID="32d076ef3ea2298b44c2d508ae1696f44fa46bc29914f84fbe2ab87735c240f5"
Oct 30 19:52:34.409242 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:34.409226    5907 scope.go:122] "RemoveContainer" containerID="f2b5d3faef45ac2468ab6e4771cc113d6f2c70a0c9f6a6dd85a1dbab4c4bacdb"
Oct 30 19:52:34.417194 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:34.417176    5907 scope.go:122] "RemoveContainer" containerID="b0c8803df220726b8f52408e2436d652f70252dda2740e4e7c73c970ae1bfc57"
Oct 30 19:52:34.425086 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:34.425068    5907 scope.go:122] "RemoveContainer" containerID="741980e41e22869eff3d026aa4c5fec6a774ad5f51aa951ebd0391ba76c5d765"
Oct 30 19:52:34.433180 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:34.433165    5907 scope.go:122] "RemoveContainer" containerID="9ad730b8879252193306fda0400eb5bc2fb3dc31dd165c70286d7e48dc7184e0"
Oct 30 19:52:34.441079 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:34.441063    5907 scope.go:122] "RemoveContainer" containerID="8954d56c4cce96b4f6365487cebef3486212fa77b07ee0ecd9683d879f70ff8a"
Oct 30 19:52:34.448627 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:34.448610    5907 scope.go:122] "RemoveContainer" containerID="910b58dfdad2ebace5d00060d016f72634c2009a047ead79e973555f8a65e1ec"
Oct 30 19:52:34.579850 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:34.579811    5907 generic.go:356] "Generic (PLEG): container finished" podID="0db10165-3412-4936-86b2-a7ec5e258915" containerID="1c46fad432ed6a4989fca4ab03e3db0cf49ab26c9ffbedb5db1c11c5717f3fa9" exitCode=0
Oct 30 19:52:34.580270 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:34.579893    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-tg6wg" event={"ID":"0db10165-3412-4936-86b2-a7ec5e258915","Type":"ContainerDied","Data":"1c46fad432ed6a4989fca4ab03e3db0cf49ab26c9ffbedb5db1c11c5717f3fa9"}
Oct 30 19:52:35.727302 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:35.727279    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-tg6wg"
Oct 30 19:52:35.766357 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:35.766328    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0db10165-3412-4936-86b2-a7ec5e258915-kube-api-access-k7c8x\" (UniqueName: \"kubernetes.io/projected/0db10165-3412-4936-86b2-a7ec5e258915-kube-api-access-k7c8x\") pod \"0db10165-3412-4936-86b2-a7ec5e258915\" (UID: \"0db10165-3412-4936-86b2-a7ec5e258915\") "
Oct 30 19:52:35.766511 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:35.766438    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0db10165-3412-4936-86b2-a7ec5e258915-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0db10165-3412-4936-86b2-a7ec5e258915-maas-api-xks-ca\") pod \"0db10165-3412-4936-86b2-a7ec5e258915\" (UID: \"0db10165-3412-4936-86b2-a7ec5e258915\") "
Oct 30 19:52:35.766511 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:35.766467    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0db10165-3412-4936-86b2-a7ec5e258915-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0db10165-3412-4936-86b2-a7ec5e258915-maas-api-openshift-ca\") pod \"0db10165-3412-4936-86b2-a7ec5e258915\" (UID: \"0db10165-3412-4936-86b2-a7ec5e258915\") "
Oct 30 19:52:35.766953 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:35.766928    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0db10165-3412-4936-86b2-a7ec5e258915-maas-api-openshift-ca" pod "0db10165-3412-4936-86b2-a7ec5e258915" (UID: "0db10165-3412-4936-86b2-a7ec5e258915"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:52:35.768855 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:35.768827    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0db10165-3412-4936-86b2-a7ec5e258915-kube-api-access-k7c8x" pod "0db10165-3412-4936-86b2-a7ec5e258915" (UID: "0db10165-3412-4936-86b2-a7ec5e258915"). InnerVolumeSpecName "kube-api-access-k7c8x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:52:35.769509 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:35.769480    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0db10165-3412-4936-86b2-a7ec5e258915-maas-api-xks-ca" pod "0db10165-3412-4936-86b2-a7ec5e258915" (UID: "0db10165-3412-4936-86b2-a7ec5e258915"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:52:35.867850 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:35.867825    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0db10165-3412-4936-86b2-a7ec5e258915-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:52:35.868005 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:35.867854    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0db10165-3412-4936-86b2-a7ec5e258915-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:52:35.868005 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:35.867886    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7c8x\" (UniqueName: \"kubernetes.io/projected/0db10165-3412-4936-86b2-a7ec5e258915-kube-api-access-k7c8x\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:52:36.593419 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:36.593332    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-tg6wg"
Oct 30 19:52:36.593604 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:36.593338    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-tg6wg" event={"ID":"0db10165-3412-4936-86b2-a7ec5e258915","Type":"ContainerDied","Data":"1a19677abfe5eaa2913da047f165593316078ae7acaf01bf1a23d7d9d0443b1a"}
Oct 30 19:52:36.593604 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:36.593460    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1a19677abfe5eaa2913da047f165593316078ae7acaf01bf1a23d7d9d0443b1a"
Oct 30 19:52:37.034470 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:37.034432    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-112c28ee-bmzsd"]
Oct 30 19:52:37.047930 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:37.047898    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d04d31e9-592l9"]
Oct 30 19:52:37.049639 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:37.049627    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-112c28ee-bmzsd"]
Oct 30 19:52:37.051510 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:37.051483    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d04d31e9-592l9"]
Oct 30 19:52:38.287034 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:38.287001    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7760da1a-a95e-4f3e-9512-08cb4e523cd7" path="/var/lib/kubelet/pods/7760da1a-a95e-4f3e-9512-08cb4e523cd7/volumes"
Oct 30 19:52:38.287445 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:38.287429    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b2fa1172-31da-4cab-9622-f4adc52c0ca2" path="/var/lib/kubelet/pods/b2fa1172-31da-4cab-9622-f4adc52c0ca2/volumes"
Oct 30 19:52:43.665165 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:43.665135    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:52:47.634040 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:52:47.634001    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:52:48.685887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:48.684976    5907 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-s826m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:52:48.685887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:48.685073    5907 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s826m" podUID="86efaf93-7f28-4e6a-9add-c8bdd87c5261" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:52:48.894892 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:48.888702    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:52:50.028242 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:50.028207    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3a667ca0-4bxzw"]
Oct 30 19:52:50.030477 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:50.030449    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3a667ca0-4bxzw"]
Oct 30 19:52:50.032892 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:50.032845    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1342a88e-rbf8h"]
Oct 30 19:52:50.035369 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:50.035348    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1342a88e-rbf8h"]
Oct 30 19:52:50.287100 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:50.287025    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1583ab53-3fec-484a-92bc-14a5b0a7ac75" path="/var/lib/kubelet/pods/1583ab53-3fec-484a-92bc-14a5b0a7ac75/volumes"
Oct 30 19:52:50.287458 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:50.287446    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dc3f5806-2a1a-4391-b578-f2885bff86f9" path="/var/lib/kubelet/pods/dc3f5806-2a1a-4391-b578-f2885bff86f9/volumes"
Oct 30 19:52:53.776652 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:53.776619    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-x7q49"]
Oct 30 19:52:53.777100 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:53.777086    5907 cpu_manager.go:414] "containerMap: removing container" podUID="0db10165-3412-4936-86b2-a7ec5e258915" containerName="revoke-keys"
Oct 30 19:52:53.777149 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:53.777101    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0db10165-3412-4936-86b2-a7ec5e258915" containerName="revoke-keys"
Oct 30 19:52:53.777220 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:53.777210    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="0db10165-3412-4936-86b2-a7ec5e258915" containerName="revoke-keys"
Oct 30 19:52:53.781171 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:53.781148    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-x7q49"
Oct 30 19:52:53.800732 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:53.800705    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:52:53.800921 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:53.800808    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:52:53.803608 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:53.803581    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-x7q49"]
Oct 30 19:52:53.942026 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:53.941992    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a4aef7d7-x7q49\" (UID: \"5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-x7q49"
Oct 30 19:52:53.942247 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:53.942052    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a4aef7d7-x7q49\" (UID: \"5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-x7q49"
Oct 30 19:52:53.942247 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:53.942139    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvnnn\" (UniqueName: \"kubernetes.io/projected/5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e-kube-api-access-cvnnn\") pod \"maas-api-revoke-sub-a4aef7d7-x7q49\" (UID: \"5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-x7q49"
Oct 30 19:52:54.043205 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:54.043127    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a4aef7d7-x7q49\" (UID: \"5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-x7q49"
Oct 30 19:52:54.043205 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:54.043181    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a4aef7d7-x7q49\" (UID: \"5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-x7q49"
Oct 30 19:52:54.043430 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:54.043313    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvnnn\" (UniqueName: \"kubernetes.io/projected/5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e-kube-api-access-cvnnn\") pod \"maas-api-revoke-sub-a4aef7d7-x7q49\" (UID: \"5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-x7q49"
Oct 30 19:52:54.043809 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:54.043791    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a4aef7d7-x7q49\" (UID: \"5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-x7q49"
Oct 30 19:52:54.045605 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:54.045584    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a4aef7d7-x7q49\" (UID: \"5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-x7q49"
Oct 30 19:52:54.064410 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:54.064389    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvnnn\" (UniqueName: \"kubernetes.io/projected/5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e-kube-api-access-cvnnn\") pod \"maas-api-revoke-sub-a4aef7d7-x7q49\" (UID: \"5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-x7q49"
Oct 30 19:52:54.095036 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:54.095010    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-x7q49"
Oct 30 19:52:54.397637 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:54.397616    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-x7q49"]
Oct 30 19:52:54.399894 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:52:54.399853    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ba41ab1_4e12_4182_b1cc_aa5fc7e7af1e.slice/crio-99e018dc78a273a45e09f4c19a6e0ad9e58b262e33e606a66a29cc0aa074f19b WatchSource:0}: Error finding container 99e018dc78a273a45e09f4c19a6e0ad9e58b262e33e606a66a29cc0aa074f19b: Status 404 returned error can't find the container with id 99e018dc78a273a45e09f4c19a6e0ad9e58b262e33e606a66a29cc0aa074f19b
Oct 30 19:52:54.740434 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:54.740399    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-x7q49" event={"ID":"5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e","Type":"ContainerStarted","Data":"503f3dfcc101693a55a839ca553a59ff7817d21d317935d920caf1608e5448c4"}
Oct 30 19:52:54.740542 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:54.740459    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-x7q49" event={"ID":"5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e","Type":"ContainerStarted","Data":"99e018dc78a273a45e09f4c19a6e0ad9e58b262e33e606a66a29cc0aa074f19b"}
Oct 30 19:52:54.781167 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:54.781114    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-x7q49" podStartSLOduration=1.7810986770000001 podStartE2EDuration="1.781098677s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:52:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:52:54.780797961 +0000 UTC m=+1886.623729899" watchObservedRunningTime="2026-10-30 19:52:54.781098677 +0000 UTC m=+1886.624030618"
Oct 30 19:52:55.747466 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:55.747432    5907 generic.go:356] "Generic (PLEG): container finished" podID="5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e" containerID="503f3dfcc101693a55a839ca553a59ff7817d21d317935d920caf1608e5448c4" exitCode=0
Oct 30 19:52:55.747656 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:55.747517    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-x7q49" event={"ID":"5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e","Type":"ContainerDied","Data":"503f3dfcc101693a55a839ca553a59ff7817d21d317935d920caf1608e5448c4"}
Oct 30 19:52:56.900994 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:56.900967    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-x7q49"
Oct 30 19:52:57.074411 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:57.074329    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e-kube-api-access-cvnnn\" (UniqueName: \"kubernetes.io/projected/5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e-kube-api-access-cvnnn\") pod \"5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e\" (UID: \"5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e\") "
Oct 30 19:52:57.074411 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:57.074368    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e-maas-api-openshift-ca\") pod \"5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e\" (UID: \"5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e\") "
Oct 30 19:52:57.074635 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:57.074482    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e-maas-api-xks-ca\") pod \"5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e\" (UID: \"5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e\") "
Oct 30 19:52:57.074773 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:57.074745    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e-maas-api-openshift-ca" pod "5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e" (UID: "5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:52:57.075011 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:57.074993    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:52:57.076657 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:57.076634    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e-maas-api-xks-ca" pod "5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e" (UID: "5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:52:57.076758 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:57.076720    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e-kube-api-access-cvnnn" pod "5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e" (UID: "5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e"). InnerVolumeSpecName "kube-api-access-cvnnn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:52:57.175640 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:57.175609    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvnnn\" (UniqueName: \"kubernetes.io/projected/5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e-kube-api-access-cvnnn\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:52:57.175640 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:57.175638    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:52:57.765831 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:57.765791    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-x7q49"
Oct 30 19:52:57.766040 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:57.765824    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-x7q49" event={"ID":"5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e","Type":"ContainerDied","Data":"99e018dc78a273a45e09f4c19a6e0ad9e58b262e33e606a66a29cc0aa074f19b"}
Oct 30 19:52:57.766040 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:52:57.765913    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="99e018dc78a273a45e09f4c19a6e0ad9e58b262e33e606a66a29cc0aa074f19b"
Oct 30 19:53:04.033048 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:04.033016    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d20ed19a-9btfj"]
Oct 30 19:53:04.046191 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:04.046165    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d20ed19a-9btfj"]
Oct 30 19:53:04.286613 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:04.286528    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fb5c110f-658c-46af-93b1-2122cc7b37b0" path="/var/lib/kubelet/pods/fb5c110f-658c-46af-93b1-2122cc7b37b0/volumes"
Oct 30 19:53:05.213894 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:05.213846    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:53:10.055501 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:10.055415    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:53:13.919819 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:13.919786    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-d5fwl"]
Oct 30 19:53:13.920287 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:13.920273    5907 cpu_manager.go:414] "containerMap: removing container" podUID="5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e" containerName="revoke-keys"
Oct 30 19:53:13.920340 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:13.920287    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e" containerName="revoke-keys"
Oct 30 19:53:13.920378 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:13.920367    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e" containerName="revoke-keys"
Oct 30 19:53:13.924911 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:13.924883    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-d5fwl"
Oct 30 19:53:13.946663 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:13.946641    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:53:13.951987 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:13.951964    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-d5fwl"]
Oct 30 19:53:13.976409 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:13.976380    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d28e37cc-059c-4e37-9d1f-5517215b3a73-maas-api-xks-ca\") pod \"maas-api-revoke-sub-79805ca9-d5fwl\" (UID: \"d28e37cc-059c-4e37-9d1f-5517215b3a73\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-d5fwl"
Oct 30 19:53:13.976545 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:13.976430    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98zjp\" (UniqueName: \"kubernetes.io/projected/d28e37cc-059c-4e37-9d1f-5517215b3a73-kube-api-access-98zjp\") pod \"maas-api-revoke-sub-79805ca9-d5fwl\" (UID: \"d28e37cc-059c-4e37-9d1f-5517215b3a73\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-d5fwl"
Oct 30 19:53:13.976656 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:13.976631    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d28e37cc-059c-4e37-9d1f-5517215b3a73-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-79805ca9-d5fwl\" (UID: \"d28e37cc-059c-4e37-9d1f-5517215b3a73\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-d5fwl"
Oct 30 19:53:14.046711 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:14.046680    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:53:14.077962 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:14.077931    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d28e37cc-059c-4e37-9d1f-5517215b3a73-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-79805ca9-d5fwl\" (UID: \"d28e37cc-059c-4e37-9d1f-5517215b3a73\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-d5fwl"
Oct 30 19:53:14.078107 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:14.077996    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d28e37cc-059c-4e37-9d1f-5517215b3a73-maas-api-xks-ca\") pod \"maas-api-revoke-sub-79805ca9-d5fwl\" (UID: \"d28e37cc-059c-4e37-9d1f-5517215b3a73\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-d5fwl"
Oct 30 19:53:14.078107 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:14.078024    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98zjp\" (UniqueName: \"kubernetes.io/projected/d28e37cc-059c-4e37-9d1f-5517215b3a73-kube-api-access-98zjp\") pod \"maas-api-revoke-sub-79805ca9-d5fwl\" (UID: \"d28e37cc-059c-4e37-9d1f-5517215b3a73\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-d5fwl"
Oct 30 19:53:14.078612 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:14.078588    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d28e37cc-059c-4e37-9d1f-5517215b3a73-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-79805ca9-d5fwl\" (UID: \"d28e37cc-059c-4e37-9d1f-5517215b3a73\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-d5fwl"
Oct 30 19:53:14.080425 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:14.080403    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d28e37cc-059c-4e37-9d1f-5517215b3a73-maas-api-xks-ca\") pod \"maas-api-revoke-sub-79805ca9-d5fwl\" (UID: \"d28e37cc-059c-4e37-9d1f-5517215b3a73\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-d5fwl"
Oct 30 19:53:14.105823 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:14.105792    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98zjp\" (UniqueName: \"kubernetes.io/projected/d28e37cc-059c-4e37-9d1f-5517215b3a73-kube-api-access-98zjp\") pod \"maas-api-revoke-sub-79805ca9-d5fwl\" (UID: \"d28e37cc-059c-4e37-9d1f-5517215b3a73\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-d5fwl"
Oct 30 19:53:14.238119 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:14.238034    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-d5fwl"
Oct 30 19:53:14.543995 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:14.543967    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-d5fwl"]
Oct 30 19:53:14.546318 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:53:14.546290    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd28e37cc_059c_4e37_9d1f_5517215b3a73.slice/crio-4f08fb26725bd89657ac7a33a16bc0e3514309cbc4c9b32f238f453965df7e46 WatchSource:0}: Error finding container 4f08fb26725bd89657ac7a33a16bc0e3514309cbc4c9b32f238f453965df7e46: Status 404 returned error can't find the container with id 4f08fb26725bd89657ac7a33a16bc0e3514309cbc4c9b32f238f453965df7e46
Oct 30 19:53:14.890882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:14.890821    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-d5fwl" event={"ID":"d28e37cc-059c-4e37-9d1f-5517215b3a73","Type":"ContainerStarted","Data":"59364f99a98c671279f88700ba4fd2bafdfcb70f758050e0d72153afe2ca9b0a"}
Oct 30 19:53:14.891068 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:14.890902    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-d5fwl" event={"ID":"d28e37cc-059c-4e37-9d1f-5517215b3a73","Type":"ContainerStarted","Data":"4f08fb26725bd89657ac7a33a16bc0e3514309cbc4c9b32f238f453965df7e46"}
Oct 30 19:53:15.898402 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:15.898367    5907 generic.go:356] "Generic (PLEG): container finished" podID="d28e37cc-059c-4e37-9d1f-5517215b3a73" containerID="59364f99a98c671279f88700ba4fd2bafdfcb70f758050e0d72153afe2ca9b0a" exitCode=0
Oct 30 19:53:15.898806 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:15.898447    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-d5fwl" event={"ID":"d28e37cc-059c-4e37-9d1f-5517215b3a73","Type":"ContainerDied","Data":"59364f99a98c671279f88700ba4fd2bafdfcb70f758050e0d72153afe2ca9b0a"}
Oct 30 19:53:17.041259 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:17.041233    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-d5fwl"
Oct 30 19:53:17.106026 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:17.105993    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d28e37cc-059c-4e37-9d1f-5517215b3a73-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d28e37cc-059c-4e37-9d1f-5517215b3a73-maas-api-openshift-ca\") pod \"d28e37cc-059c-4e37-9d1f-5517215b3a73\" (UID: \"d28e37cc-059c-4e37-9d1f-5517215b3a73\") "
Oct 30 19:53:17.106233 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:17.106039    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d28e37cc-059c-4e37-9d1f-5517215b3a73-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d28e37cc-059c-4e37-9d1f-5517215b3a73-maas-api-xks-ca\") pod \"d28e37cc-059c-4e37-9d1f-5517215b3a73\" (UID: \"d28e37cc-059c-4e37-9d1f-5517215b3a73\") "
Oct 30 19:53:17.106233 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:17.106071    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d28e37cc-059c-4e37-9d1f-5517215b3a73-kube-api-access-98zjp\" (UniqueName: \"kubernetes.io/projected/d28e37cc-059c-4e37-9d1f-5517215b3a73-kube-api-access-98zjp\") pod \"d28e37cc-059c-4e37-9d1f-5517215b3a73\" (UID: \"d28e37cc-059c-4e37-9d1f-5517215b3a73\") "
Oct 30 19:53:17.106461 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:17.106437    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d28e37cc-059c-4e37-9d1f-5517215b3a73-maas-api-openshift-ca" pod "d28e37cc-059c-4e37-9d1f-5517215b3a73" (UID: "d28e37cc-059c-4e37-9d1f-5517215b3a73"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:53:17.108429 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:17.108400    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d28e37cc-059c-4e37-9d1f-5517215b3a73-maas-api-xks-ca" pod "d28e37cc-059c-4e37-9d1f-5517215b3a73" (UID: "d28e37cc-059c-4e37-9d1f-5517215b3a73"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:53:17.108584 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:17.108567    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d28e37cc-059c-4e37-9d1f-5517215b3a73-kube-api-access-98zjp" pod "d28e37cc-059c-4e37-9d1f-5517215b3a73" (UID: "d28e37cc-059c-4e37-9d1f-5517215b3a73"). InnerVolumeSpecName "kube-api-access-98zjp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:53:17.207258 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:17.207174    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d28e37cc-059c-4e37-9d1f-5517215b3a73-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:53:17.207258 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:17.207207    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d28e37cc-059c-4e37-9d1f-5517215b3a73-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:53:17.207258 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:17.207217    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98zjp\" (UniqueName: \"kubernetes.io/projected/d28e37cc-059c-4e37-9d1f-5517215b3a73-kube-api-access-98zjp\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:53:17.499269 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:53:17.499187    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:53:17.914623 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:17.914591    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-d5fwl"
Oct 30 19:53:17.914825 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:17.914624    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-d5fwl" event={"ID":"d28e37cc-059c-4e37-9d1f-5517215b3a73","Type":"ContainerDied","Data":"4f08fb26725bd89657ac7a33a16bc0e3514309cbc4c9b32f238f453965df7e46"}
Oct 30 19:53:17.914825 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:17.914686    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4f08fb26725bd89657ac7a33a16bc0e3514309cbc4c9b32f238f453965df7e46"
Oct 30 19:53:26.150279 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:26.150244    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:53:30.810089 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:30.810048    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-kxwpx"]
Oct 30 19:53:30.810753 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:30.810733    5907 cpu_manager.go:414] "containerMap: removing container" podUID="d28e37cc-059c-4e37-9d1f-5517215b3a73" containerName="revoke-keys"
Oct 30 19:53:30.810800 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:30.810759    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d28e37cc-059c-4e37-9d1f-5517215b3a73" containerName="revoke-keys"
Oct 30 19:53:30.810960 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:30.810945    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="d28e37cc-059c-4e37-9d1f-5517215b3a73" containerName="revoke-keys"
Oct 30 19:53:30.814970 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:30.814947    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-kxwpx"
Oct 30 19:53:30.833758 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:30.833725    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-kxwpx"]
Oct 30 19:53:30.844124 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:30.844102    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:53:30.844261 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:30.844126    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:53:30.918836 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:30.918804    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsg8s\" (UniqueName: \"kubernetes.io/projected/5df7d60f-e0d7-4c92-99c9-5593d805dd51-kube-api-access-tsg8s\") pod \"maas-api-revoke-sub-0b7e8edb-kxwpx\" (UID: \"5df7d60f-e0d7-4c92-99c9-5593d805dd51\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-kxwpx"
Oct 30 19:53:30.919015 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:30.918952    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5df7d60f-e0d7-4c92-99c9-5593d805dd51-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0b7e8edb-kxwpx\" (UID: \"5df7d60f-e0d7-4c92-99c9-5593d805dd51\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-kxwpx"
Oct 30 19:53:30.919064 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:30.919019    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5df7d60f-e0d7-4c92-99c9-5593d805dd51-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0b7e8edb-kxwpx\" (UID: \"5df7d60f-e0d7-4c92-99c9-5593d805dd51\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-kxwpx"
Oct 30 19:53:31.019976 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:31.019943    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5df7d60f-e0d7-4c92-99c9-5593d805dd51-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0b7e8edb-kxwpx\" (UID: \"5df7d60f-e0d7-4c92-99c9-5593d805dd51\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-kxwpx"
Oct 30 19:53:31.020156 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:31.020012    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5df7d60f-e0d7-4c92-99c9-5593d805dd51-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0b7e8edb-kxwpx\" (UID: \"5df7d60f-e0d7-4c92-99c9-5593d805dd51\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-kxwpx"
Oct 30 19:53:31.020156 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:31.020091    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsg8s\" (UniqueName: \"kubernetes.io/projected/5df7d60f-e0d7-4c92-99c9-5593d805dd51-kube-api-access-tsg8s\") pod \"maas-api-revoke-sub-0b7e8edb-kxwpx\" (UID: \"5df7d60f-e0d7-4c92-99c9-5593d805dd51\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-kxwpx"
Oct 30 19:53:31.020613 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:31.020591    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5df7d60f-e0d7-4c92-99c9-5593d805dd51-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0b7e8edb-kxwpx\" (UID: \"5df7d60f-e0d7-4c92-99c9-5593d805dd51\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-kxwpx"
Oct 30 19:53:31.022527 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:31.022505    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5df7d60f-e0d7-4c92-99c9-5593d805dd51-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0b7e8edb-kxwpx\" (UID: \"5df7d60f-e0d7-4c92-99c9-5593d805dd51\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-kxwpx"
Oct 30 19:53:31.040193 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:31.040168    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsg8s\" (UniqueName: \"kubernetes.io/projected/5df7d60f-e0d7-4c92-99c9-5593d805dd51-kube-api-access-tsg8s\") pod \"maas-api-revoke-sub-0b7e8edb-kxwpx\" (UID: \"5df7d60f-e0d7-4c92-99c9-5593d805dd51\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-kxwpx"
Oct 30 19:53:31.128816 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:31.128790    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-kxwpx"
Oct 30 19:53:31.430851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:31.430819    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-kxwpx"]
Oct 30 19:53:31.433082 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:53:31.433053    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5df7d60f_e0d7_4c92_99c9_5593d805dd51.slice/crio-3b3a1036a32ed66b0b93c2e51fedfdeade1c2dc90200618a440fe4b56244f2c7 WatchSource:0}: Error finding container 3b3a1036a32ed66b0b93c2e51fedfdeade1c2dc90200618a440fe4b56244f2c7: Status 404 returned error can't find the container with id 3b3a1036a32ed66b0b93c2e51fedfdeade1c2dc90200618a440fe4b56244f2c7
Oct 30 19:53:32.020078 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:32.019987    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-kxwpx" event={"ID":"5df7d60f-e0d7-4c92-99c9-5593d805dd51","Type":"ContainerStarted","Data":"2ccb89b5be17451678f24c1daf17f660cc39a3dc0938e1b23dc13a560e81d06d"}
Oct 30 19:53:32.020078 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:32.020040    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-kxwpx" event={"ID":"5df7d60f-e0d7-4c92-99c9-5593d805dd51","Type":"ContainerStarted","Data":"3b3a1036a32ed66b0b93c2e51fedfdeade1c2dc90200618a440fe4b56244f2c7"}
Oct 30 19:53:32.064186 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:32.064133    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-kxwpx" podStartSLOduration=2.064118564 podStartE2EDuration="2.064118564s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:53:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:53:32.062630041 +0000 UTC m=+1923.905561979" watchObservedRunningTime="2026-10-30 19:53:32.064118564 +0000 UTC m=+1923.907050500"
Oct 30 19:53:33.027467 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:33.027431    5907 generic.go:356] "Generic (PLEG): container finished" podID="5df7d60f-e0d7-4c92-99c9-5593d805dd51" containerID="2ccb89b5be17451678f24c1daf17f660cc39a3dc0938e1b23dc13a560e81d06d" exitCode=0
Oct 30 19:53:33.027903 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:33.027502    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-kxwpx" event={"ID":"5df7d60f-e0d7-4c92-99c9-5593d805dd51","Type":"ContainerDied","Data":"2ccb89b5be17451678f24c1daf17f660cc39a3dc0938e1b23dc13a560e81d06d"}
Oct 30 19:53:34.169575 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:34.169551    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-kxwpx"
Oct 30 19:53:34.251116 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:34.251077    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5df7d60f-e0d7-4c92-99c9-5593d805dd51-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5df7d60f-e0d7-4c92-99c9-5593d805dd51-maas-api-xks-ca\") pod \"5df7d60f-e0d7-4c92-99c9-5593d805dd51\" (UID: \"5df7d60f-e0d7-4c92-99c9-5593d805dd51\") "
Oct 30 19:53:34.251386 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:34.251137    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5df7d60f-e0d7-4c92-99c9-5593d805dd51-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5df7d60f-e0d7-4c92-99c9-5593d805dd51-maas-api-openshift-ca\") pod \"5df7d60f-e0d7-4c92-99c9-5593d805dd51\" (UID: \"5df7d60f-e0d7-4c92-99c9-5593d805dd51\") "
Oct 30 19:53:34.251386 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:34.251180    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5df7d60f-e0d7-4c92-99c9-5593d805dd51-kube-api-access-tsg8s\" (UniqueName: \"kubernetes.io/projected/5df7d60f-e0d7-4c92-99c9-5593d805dd51-kube-api-access-tsg8s\") pod \"5df7d60f-e0d7-4c92-99c9-5593d805dd51\" (UID: \"5df7d60f-e0d7-4c92-99c9-5593d805dd51\") "
Oct 30 19:53:34.251585 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:34.251555    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5df7d60f-e0d7-4c92-99c9-5593d805dd51-maas-api-openshift-ca" pod "5df7d60f-e0d7-4c92-99c9-5593d805dd51" (UID: "5df7d60f-e0d7-4c92-99c9-5593d805dd51"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:53:34.253346 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:34.253323    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5df7d60f-e0d7-4c92-99c9-5593d805dd51-maas-api-xks-ca" pod "5df7d60f-e0d7-4c92-99c9-5593d805dd51" (UID: "5df7d60f-e0d7-4c92-99c9-5593d805dd51"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:53:34.253458 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:34.253377    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5df7d60f-e0d7-4c92-99c9-5593d805dd51-kube-api-access-tsg8s" pod "5df7d60f-e0d7-4c92-99c9-5593d805dd51" (UID: "5df7d60f-e0d7-4c92-99c9-5593d805dd51"). InnerVolumeSpecName "kube-api-access-tsg8s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:53:34.352299 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:34.352267    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tsg8s\" (UniqueName: \"kubernetes.io/projected/5df7d60f-e0d7-4c92-99c9-5593d805dd51-kube-api-access-tsg8s\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:53:34.352299 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:34.352297    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5df7d60f-e0d7-4c92-99c9-5593d805dd51-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:53:34.352489 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:34.352310    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5df7d60f-e0d7-4c92-99c9-5593d805dd51-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:53:34.500465 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:34.500436    5907 scope.go:122] "RemoveContainer" containerID="b25d3d304895da9e40e246b343fb784589274c89a57815380cb65691cdbaf0d8"
Oct 30 19:53:34.510424 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:34.510402    5907 scope.go:122] "RemoveContainer" containerID="15c522243d334731ab97cfb4b7ed018144321739bc94ffaa8992551c3b42699b"
Oct 30 19:53:34.518740 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:34.518723    5907 scope.go:122] "RemoveContainer" containerID="34dafadcc865c56cb5dcc851a407f88348f989c632ef48d5c528518b30827cb7"
Oct 30 19:53:34.526838 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:34.526820    5907 scope.go:122] "RemoveContainer" containerID="f9912a04ad55516eac512d90e632e752e2a09633801d47b22a7c149df41f19a6"
Oct 30 19:53:34.534560 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:34.534539    5907 scope.go:122] "RemoveContainer" containerID="8886ea31c61d5c833b77552dd9a7e8acf584ceced29aa73d466f42a52fb392ea"
Oct 30 19:53:34.542227 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:34.542208    5907 scope.go:122] "RemoveContainer" containerID="40df6df7a820baea9118e3e42586258d0c691515a33b827ede5fade8425d8a09"
Oct 30 19:53:34.549833 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:34.549816    5907 scope.go:122] "RemoveContainer" containerID="9a570bbcc2174846b7fd60475f46e6376306c410a0f0cd9c99ea2bb109516c0d"
Oct 30 19:53:34.557871 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:34.557840    5907 scope.go:122] "RemoveContainer" containerID="59d996e025b64708a0bd38f4f6a8ffeabbc15a6609d64a590e584a1ee57130de"
Oct 30 19:53:35.039589 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:35.039557    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-kxwpx"
Oct 30 19:53:35.039589 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:35.039572    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-kxwpx" event={"ID":"5df7d60f-e0d7-4c92-99c9-5593d805dd51","Type":"ContainerDied","Data":"3b3a1036a32ed66b0b93c2e51fedfdeade1c2dc90200618a440fe4b56244f2c7"}
Oct 30 19:53:35.039911 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:35.039644    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3b3a1036a32ed66b0b93c2e51fedfdeade1c2dc90200618a440fe4b56244f2c7"
Oct 30 19:53:41.593799 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:41.593765    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-cjqqb"]
Oct 30 19:53:41.594559 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:41.594225    5907 cpu_manager.go:414] "containerMap: removing container" podUID="5df7d60f-e0d7-4c92-99c9-5593d805dd51" containerName="revoke-keys"
Oct 30 19:53:41.594559 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:41.594237    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5df7d60f-e0d7-4c92-99c9-5593d805dd51" containerName="revoke-keys"
Oct 30 19:53:41.597107 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:41.595233    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="5df7d60f-e0d7-4c92-99c9-5593d805dd51" containerName="revoke-keys"
Oct 30 19:53:41.606519 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:41.606493    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-cjqqb"
Oct 30 19:53:41.621730 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:41.621686    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-cjqqb"]
Oct 30 19:53:41.630235 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:41.630208    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:53:41.630437 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:41.630271    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:53:41.726880 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:41.726829    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzxp6\" (UniqueName: \"kubernetes.io/projected/96d8c020-aa60-4b69-9b2a-933e02685887-kube-api-access-qzxp6\") pod \"maas-api-revoke-sub-3d4156f1-cjqqb\" (UID: \"96d8c020-aa60-4b69-9b2a-933e02685887\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-cjqqb"
Oct 30 19:53:41.727060 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:41.726954    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/96d8c020-aa60-4b69-9b2a-933e02685887-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-cjqqb\" (UID: \"96d8c020-aa60-4b69-9b2a-933e02685887\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-cjqqb"
Oct 30 19:53:41.727107 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:41.727061    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/96d8c020-aa60-4b69-9b2a-933e02685887-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-cjqqb\" (UID: \"96d8c020-aa60-4b69-9b2a-933e02685887\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-cjqqb"
Oct 30 19:53:41.828253 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:41.828216    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/96d8c020-aa60-4b69-9b2a-933e02685887-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-cjqqb\" (UID: \"96d8c020-aa60-4b69-9b2a-933e02685887\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-cjqqb"
Oct 30 19:53:41.828441 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:41.828301    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzxp6\" (UniqueName: \"kubernetes.io/projected/96d8c020-aa60-4b69-9b2a-933e02685887-kube-api-access-qzxp6\") pod \"maas-api-revoke-sub-3d4156f1-cjqqb\" (UID: \"96d8c020-aa60-4b69-9b2a-933e02685887\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-cjqqb"
Oct 30 19:53:41.828441 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:41.828344    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/96d8c020-aa60-4b69-9b2a-933e02685887-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-cjqqb\" (UID: \"96d8c020-aa60-4b69-9b2a-933e02685887\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-cjqqb"
Oct 30 19:53:41.829060 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:41.829030    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/96d8c020-aa60-4b69-9b2a-933e02685887-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-cjqqb\" (UID: \"96d8c020-aa60-4b69-9b2a-933e02685887\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-cjqqb"
Oct 30 19:53:41.830764 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:41.830735    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/96d8c020-aa60-4b69-9b2a-933e02685887-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-cjqqb\" (UID: \"96d8c020-aa60-4b69-9b2a-933e02685887\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-cjqqb"
Oct 30 19:53:41.851327 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:41.851277    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzxp6\" (UniqueName: \"kubernetes.io/projected/96d8c020-aa60-4b69-9b2a-933e02685887-kube-api-access-qzxp6\") pod \"maas-api-revoke-sub-3d4156f1-cjqqb\" (UID: \"96d8c020-aa60-4b69-9b2a-933e02685887\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-cjqqb"
Oct 30 19:53:41.924034 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:41.924006    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-cjqqb"
Oct 30 19:53:42.176325 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:42.176295    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:53:42.230364 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:42.230343    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-cjqqb"]
Oct 30 19:53:42.233679 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:53:42.233650    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96d8c020_aa60_4b69_9b2a_933e02685887.slice/crio-8542a4ef006e3ede17d4da395248c96c397b81560cc7342b167f16c4a25e36da WatchSource:0}: Error finding container 8542a4ef006e3ede17d4da395248c96c397b81560cc7342b167f16c4a25e36da: Status 404 returned error can't find the container with id 8542a4ef006e3ede17d4da395248c96c397b81560cc7342b167f16c4a25e36da
Oct 30 19:53:43.084610 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:43.084565    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-cjqqb" event={"ID":"96d8c020-aa60-4b69-9b2a-933e02685887","Type":"ContainerStarted","Data":"6eb6819a8ddbb33410584fb85212c95325424dd65b145428fffa1b0a4aaeb055"}
Oct 30 19:53:43.085069 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:43.084630    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-cjqqb" event={"ID":"96d8c020-aa60-4b69-9b2a-933e02685887","Type":"ContainerStarted","Data":"8542a4ef006e3ede17d4da395248c96c397b81560cc7342b167f16c4a25e36da"}
Oct 30 19:53:43.127568 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:43.127517    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-cjqqb" podStartSLOduration=2.127501564 podStartE2EDuration="2.127501564s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:53:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:53:43.126175047 +0000 UTC m=+1934.969106991" watchObservedRunningTime="2026-10-30 19:53:43.127501564 +0000 UTC m=+1934.970433500"
Oct 30 19:53:44.091778 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:44.091740    5907 generic.go:356] "Generic (PLEG): container finished" podID="96d8c020-aa60-4b69-9b2a-933e02685887" containerID="6eb6819a8ddbb33410584fb85212c95325424dd65b145428fffa1b0a4aaeb055" exitCode=0
Oct 30 19:53:44.092226 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:44.091821    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-cjqqb" event={"ID":"96d8c020-aa60-4b69-9b2a-933e02685887","Type":"ContainerDied","Data":"6eb6819a8ddbb33410584fb85212c95325424dd65b145428fffa1b0a4aaeb055"}
Oct 30 19:53:45.235035 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:45.235008    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-cjqqb"
Oct 30 19:53:45.366276 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:45.366247    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/96d8c020-aa60-4b69-9b2a-933e02685887-kube-api-access-qzxp6\" (UniqueName: \"kubernetes.io/projected/96d8c020-aa60-4b69-9b2a-933e02685887-kube-api-access-qzxp6\") pod \"96d8c020-aa60-4b69-9b2a-933e02685887\" (UID: \"96d8c020-aa60-4b69-9b2a-933e02685887\") "
Oct 30 19:53:45.366466 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:45.366293    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/96d8c020-aa60-4b69-9b2a-933e02685887-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/96d8c020-aa60-4b69-9b2a-933e02685887-maas-api-xks-ca\") pod \"96d8c020-aa60-4b69-9b2a-933e02685887\" (UID: \"96d8c020-aa60-4b69-9b2a-933e02685887\") "
Oct 30 19:53:45.366466 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:45.366372    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/96d8c020-aa60-4b69-9b2a-933e02685887-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/96d8c020-aa60-4b69-9b2a-933e02685887-maas-api-openshift-ca\") pod \"96d8c020-aa60-4b69-9b2a-933e02685887\" (UID: \"96d8c020-aa60-4b69-9b2a-933e02685887\") "
Oct 30 19:53:45.366813 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:45.366778    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96d8c020-aa60-4b69-9b2a-933e02685887-maas-api-openshift-ca" pod "96d8c020-aa60-4b69-9b2a-933e02685887" (UID: "96d8c020-aa60-4b69-9b2a-933e02685887"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:53:45.368538 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:45.368514    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96d8c020-aa60-4b69-9b2a-933e02685887-kube-api-access-qzxp6" pod "96d8c020-aa60-4b69-9b2a-933e02685887" (UID: "96d8c020-aa60-4b69-9b2a-933e02685887"). InnerVolumeSpecName "kube-api-access-qzxp6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:53:45.368649 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:45.368585    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96d8c020-aa60-4b69-9b2a-933e02685887-maas-api-xks-ca" pod "96d8c020-aa60-4b69-9b2a-933e02685887" (UID: "96d8c020-aa60-4b69-9b2a-933e02685887"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:53:45.468237 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:45.468204    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/96d8c020-aa60-4b69-9b2a-933e02685887-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:53:45.468237 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:45.468234    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qzxp6\" (UniqueName: \"kubernetes.io/projected/96d8c020-aa60-4b69-9b2a-933e02685887-kube-api-access-qzxp6\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:53:45.468237 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:45.468244    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/96d8c020-aa60-4b69-9b2a-933e02685887-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:53:46.104133 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:46.104105    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-cjqqb"
Oct 30 19:53:46.104133 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:46.104141    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-cjqqb" event={"ID":"96d8c020-aa60-4b69-9b2a-933e02685887","Type":"ContainerDied","Data":"8542a4ef006e3ede17d4da395248c96c397b81560cc7342b167f16c4a25e36da"}
Oct 30 19:53:46.104133 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:46.104187    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8542a4ef006e3ede17d4da395248c96c397b81560cc7342b167f16c4a25e36da"
Oct 30 19:53:47.498775 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:53:47.498729    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:53:52.875658 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:52.875626    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:53:58.893682 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:53:58.887769    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:54:02.894665 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:02.894626    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-cjqqb"]
Oct 30 19:54:02.907744 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:02.907716    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-cjqqb"]
Oct 30 19:54:02.940107 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:02.940071    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pc86p"]
Oct 30 19:54:02.940570 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:02.940556    5907 cpu_manager.go:414] "containerMap: removing container" podUID="96d8c020-aa60-4b69-9b2a-933e02685887" containerName="revoke-keys"
Oct 30 19:54:02.940612 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:02.940574    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="96d8c020-aa60-4b69-9b2a-933e02685887" containerName="revoke-keys"
Oct 30 19:54:02.940694 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:02.940682    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="96d8c020-aa60-4b69-9b2a-933e02685887" containerName="revoke-keys"
Oct 30 19:54:02.944322 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:02.944304    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pc86p"
Oct 30 19:54:02.961847 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:02.961818    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:54:02.962051 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:02.961908    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:54:02.966250 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:02.966220    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pc86p"]
Oct 30 19:54:03.024150 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:03.024102    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/019e0695-5b4c-4efe-8360-a128b10e60e0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-pc86p\" (UID: \"019e0695-5b4c-4efe-8360-a128b10e60e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pc86p"
Oct 30 19:54:03.024340 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:03.024250    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kpjl\" (UniqueName: \"kubernetes.io/projected/019e0695-5b4c-4efe-8360-a128b10e60e0-kube-api-access-8kpjl\") pod \"maas-api-revoke-sub-3d4156f1-pc86p\" (UID: \"019e0695-5b4c-4efe-8360-a128b10e60e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pc86p"
Oct 30 19:54:03.024340 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:03.024283    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/019e0695-5b4c-4efe-8360-a128b10e60e0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-pc86p\" (UID: \"019e0695-5b4c-4efe-8360-a128b10e60e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pc86p"
Oct 30 19:54:03.125663 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:03.125627    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kpjl\" (UniqueName: \"kubernetes.io/projected/019e0695-5b4c-4efe-8360-a128b10e60e0-kube-api-access-8kpjl\") pod \"maas-api-revoke-sub-3d4156f1-pc86p\" (UID: \"019e0695-5b4c-4efe-8360-a128b10e60e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pc86p"
Oct 30 19:54:03.125832 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:03.125685    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/019e0695-5b4c-4efe-8360-a128b10e60e0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-pc86p\" (UID: \"019e0695-5b4c-4efe-8360-a128b10e60e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pc86p"
Oct 30 19:54:03.125832 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:03.125802    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/019e0695-5b4c-4efe-8360-a128b10e60e0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-pc86p\" (UID: \"019e0695-5b4c-4efe-8360-a128b10e60e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pc86p"
Oct 30 19:54:03.126455 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:03.126425    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/019e0695-5b4c-4efe-8360-a128b10e60e0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-pc86p\" (UID: \"019e0695-5b4c-4efe-8360-a128b10e60e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pc86p"
Oct 30 19:54:03.128256 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:03.128236    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/019e0695-5b4c-4efe-8360-a128b10e60e0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-pc86p\" (UID: \"019e0695-5b4c-4efe-8360-a128b10e60e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pc86p"
Oct 30 19:54:03.146012 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:03.145961    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kpjl\" (UniqueName: \"kubernetes.io/projected/019e0695-5b4c-4efe-8360-a128b10e60e0-kube-api-access-8kpjl\") pod \"maas-api-revoke-sub-3d4156f1-pc86p\" (UID: \"019e0695-5b4c-4efe-8360-a128b10e60e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pc86p"
Oct 30 19:54:03.257442 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:03.257414    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pc86p"
Oct 30 19:54:03.561647 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:03.561619    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pc86p"]
Oct 30 19:54:03.563885 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:54:03.563841    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod019e0695_5b4c_4efe_8360_a128b10e60e0.slice/crio-367e49a2f19c84d3a27acb21de3049468d2d1b45c8a11e9d706afef1e083f751 WatchSource:0}: Error finding container 367e49a2f19c84d3a27acb21de3049468d2d1b45c8a11e9d706afef1e083f751: Status 404 returned error can't find the container with id 367e49a2f19c84d3a27acb21de3049468d2d1b45c8a11e9d706afef1e083f751
Oct 30 19:54:04.247753 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:04.247712    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pc86p" event={"ID":"019e0695-5b4c-4efe-8360-a128b10e60e0","Type":"ContainerStarted","Data":"4ce8016ba076c1d6436ba048cbd94144f28fddd306621a53e3a4676fb51f96c9"}
Oct 30 19:54:04.248358 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:04.247768    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pc86p" event={"ID":"019e0695-5b4c-4efe-8360-a128b10e60e0","Type":"ContainerStarted","Data":"367e49a2f19c84d3a27acb21de3049468d2d1b45c8a11e9d706afef1e083f751"}
Oct 30 19:54:04.286398 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:04.286363    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="96d8c020-aa60-4b69-9b2a-933e02685887" path="/var/lib/kubelet/pods/96d8c020-aa60-4b69-9b2a-933e02685887/volumes"
Oct 30 19:54:04.290222 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:04.290175    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pc86p" podStartSLOduration=2.29016206 podStartE2EDuration="2.29016206s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:54:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:54:04.288117075 +0000 UTC m=+1956.131049014" watchObservedRunningTime="2026-10-30 19:54:04.29016206 +0000 UTC m=+1956.133093999"
Oct 30 19:54:05.254810 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:05.254777    5907 generic.go:356] "Generic (PLEG): container finished" podID="019e0695-5b4c-4efe-8360-a128b10e60e0" containerID="4ce8016ba076c1d6436ba048cbd94144f28fddd306621a53e3a4676fb51f96c9" exitCode=0
Oct 30 19:54:05.255265 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:05.254818    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pc86p" event={"ID":"019e0695-5b4c-4efe-8360-a128b10e60e0","Type":"ContainerDied","Data":"4ce8016ba076c1d6436ba048cbd94144f28fddd306621a53e3a4676fb51f96c9"}
Oct 30 19:54:06.399680 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:06.399651    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pc86p"
Oct 30 19:54:06.458178 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:06.458146    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/019e0695-5b4c-4efe-8360-a128b10e60e0-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/019e0695-5b4c-4efe-8360-a128b10e60e0-maas-api-openshift-ca\") pod \"019e0695-5b4c-4efe-8360-a128b10e60e0\" (UID: \"019e0695-5b4c-4efe-8360-a128b10e60e0\") "
Oct 30 19:54:06.458347 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:06.458204    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/019e0695-5b4c-4efe-8360-a128b10e60e0-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/019e0695-5b4c-4efe-8360-a128b10e60e0-maas-api-xks-ca\") pod \"019e0695-5b4c-4efe-8360-a128b10e60e0\" (UID: \"019e0695-5b4c-4efe-8360-a128b10e60e0\") "
Oct 30 19:54:06.458427 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:06.458393    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/019e0695-5b4c-4efe-8360-a128b10e60e0-kube-api-access-8kpjl\" (UniqueName: \"kubernetes.io/projected/019e0695-5b4c-4efe-8360-a128b10e60e0-kube-api-access-8kpjl\") pod \"019e0695-5b4c-4efe-8360-a128b10e60e0\" (UID: \"019e0695-5b4c-4efe-8360-a128b10e60e0\") "
Oct 30 19:54:06.458552 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:06.458524    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/019e0695-5b4c-4efe-8360-a128b10e60e0-maas-api-openshift-ca" pod "019e0695-5b4c-4efe-8360-a128b10e60e0" (UID: "019e0695-5b4c-4efe-8360-a128b10e60e0"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:54:06.458753 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:06.458730    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/019e0695-5b4c-4efe-8360-a128b10e60e0-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:54:06.460549 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:06.460525    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/019e0695-5b4c-4efe-8360-a128b10e60e0-maas-api-xks-ca" pod "019e0695-5b4c-4efe-8360-a128b10e60e0" (UID: "019e0695-5b4c-4efe-8360-a128b10e60e0"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:54:06.462058 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:06.462038    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/019e0695-5b4c-4efe-8360-a128b10e60e0-kube-api-access-8kpjl" pod "019e0695-5b4c-4efe-8360-a128b10e60e0" (UID: "019e0695-5b4c-4efe-8360-a128b10e60e0"). InnerVolumeSpecName "kube-api-access-8kpjl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:54:06.560093 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:06.560016    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8kpjl\" (UniqueName: \"kubernetes.io/projected/019e0695-5b4c-4efe-8360-a128b10e60e0-kube-api-access-8kpjl\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:54:06.560093 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:06.560046    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/019e0695-5b4c-4efe-8360-a128b10e60e0-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:54:07.267165 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:07.267129    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pc86p"
Oct 30 19:54:07.267351 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:07.267163    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pc86p" event={"ID":"019e0695-5b4c-4efe-8360-a128b10e60e0","Type":"ContainerDied","Data":"367e49a2f19c84d3a27acb21de3049468d2d1b45c8a11e9d706afef1e083f751"}
Oct 30 19:54:07.267351 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:07.267229    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="367e49a2f19c84d3a27acb21de3049468d2d1b45c8a11e9d706afef1e083f751"
Oct 30 19:54:10.030891 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:10.030845    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7762c4d1-6rr8x"]
Oct 30 19:54:10.044002 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:10.043968    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7762c4d1-6rr8x"]
Oct 30 19:54:10.286706 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:10.286622    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4" path="/var/lib/kubelet/pods/4bd53c00-d91a-4fc5-9a1d-8626bdbec4e4/volumes"
Oct 30 19:54:17.499726 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:54:17.499688    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:54:23.765776 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:23.765740    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:54:30.418844 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:30.418801    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kcqxp"]
Oct 30 19:54:30.419505 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:30.419467    5907 cpu_manager.go:414] "containerMap: removing container" podUID="019e0695-5b4c-4efe-8360-a128b10e60e0" containerName="revoke-keys"
Oct 30 19:54:30.419505 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:30.419488    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="019e0695-5b4c-4efe-8360-a128b10e60e0" containerName="revoke-keys"
Oct 30 19:54:30.419669 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:30.419653    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="019e0695-5b4c-4efe-8360-a128b10e60e0" containerName="revoke-keys"
Oct 30 19:54:30.422953 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:30.422932    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kcqxp"
Oct 30 19:54:30.441218 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:30.441194    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:54:30.454287 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:30.454251    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:54:30.456286 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:30.456258    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kcqxp"]
Oct 30 19:54:30.530744 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:30.530710    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krtfk\" (UniqueName: \"kubernetes.io/projected/919c056f-1a01-4b8f-a6ee-0f49f79e64ad-kube-api-access-krtfk\") pod \"maas-api-revoke-sub-e2fc9c5b-kcqxp\" (UID: \"919c056f-1a01-4b8f-a6ee-0f49f79e64ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kcqxp"
Oct 30 19:54:30.530981 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:30.530883    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/919c056f-1a01-4b8f-a6ee-0f49f79e64ad-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e2fc9c5b-kcqxp\" (UID: \"919c056f-1a01-4b8f-a6ee-0f49f79e64ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kcqxp"
Oct 30 19:54:30.530981 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:30.530973    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/919c056f-1a01-4b8f-a6ee-0f49f79e64ad-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e2fc9c5b-kcqxp\" (UID: \"919c056f-1a01-4b8f-a6ee-0f49f79e64ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kcqxp"
Oct 30 19:54:30.632151 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:30.632117    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/919c056f-1a01-4b8f-a6ee-0f49f79e64ad-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e2fc9c5b-kcqxp\" (UID: \"919c056f-1a01-4b8f-a6ee-0f49f79e64ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kcqxp"
Oct 30 19:54:30.632352 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:30.632176    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/919c056f-1a01-4b8f-a6ee-0f49f79e64ad-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e2fc9c5b-kcqxp\" (UID: \"919c056f-1a01-4b8f-a6ee-0f49f79e64ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kcqxp"
Oct 30 19:54:30.632352 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:30.632213    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krtfk\" (UniqueName: \"kubernetes.io/projected/919c056f-1a01-4b8f-a6ee-0f49f79e64ad-kube-api-access-krtfk\") pod \"maas-api-revoke-sub-e2fc9c5b-kcqxp\" (UID: \"919c056f-1a01-4b8f-a6ee-0f49f79e64ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kcqxp"
Oct 30 19:54:30.632952 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:30.632929    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/919c056f-1a01-4b8f-a6ee-0f49f79e64ad-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e2fc9c5b-kcqxp\" (UID: \"919c056f-1a01-4b8f-a6ee-0f49f79e64ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kcqxp"
Oct 30 19:54:30.634768 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:30.634740    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/919c056f-1a01-4b8f-a6ee-0f49f79e64ad-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e2fc9c5b-kcqxp\" (UID: \"919c056f-1a01-4b8f-a6ee-0f49f79e64ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kcqxp"
Oct 30 19:54:30.653598 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:30.653573    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-krtfk\" (UniqueName: \"kubernetes.io/projected/919c056f-1a01-4b8f-a6ee-0f49f79e64ad-kube-api-access-krtfk\") pod \"maas-api-revoke-sub-e2fc9c5b-kcqxp\" (UID: \"919c056f-1a01-4b8f-a6ee-0f49f79e64ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kcqxp"
Oct 30 19:54:30.736722 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:30.736643    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kcqxp"
Oct 30 19:54:31.039544 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:31.039517    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kcqxp"]
Oct 30 19:54:31.041918 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:54:31.041890    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod919c056f_1a01_4b8f_a6ee_0f49f79e64ad.slice/crio-b1165039eeae69b88f2abf1d5b58319675d4d440a628c6202df0325b27b2d0e0 WatchSource:0}: Error finding container b1165039eeae69b88f2abf1d5b58319675d4d440a628c6202df0325b27b2d0e0: Status 404 returned error can't find the container with id b1165039eeae69b88f2abf1d5b58319675d4d440a628c6202df0325b27b2d0e0
Oct 30 19:54:31.445617 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:31.445580    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kcqxp" event={"ID":"919c056f-1a01-4b8f-a6ee-0f49f79e64ad","Type":"ContainerStarted","Data":"373869832d6f1cf3614c31c80a4082c4944493c555fd16e476ee01ecba483a52"}
Oct 30 19:54:31.446053 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:31.445638    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kcqxp" event={"ID":"919c056f-1a01-4b8f-a6ee-0f49f79e64ad","Type":"ContainerStarted","Data":"b1165039eeae69b88f2abf1d5b58319675d4d440a628c6202df0325b27b2d0e0"}
Oct 30 19:54:31.493856 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:31.493805    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kcqxp" podStartSLOduration=1.49379236 podStartE2EDuration="1.49379236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:54:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:54:31.489494385 +0000 UTC m=+1983.332426324" watchObservedRunningTime="2026-10-30 19:54:31.49379236 +0000 UTC m=+1983.336724334"
Oct 30 19:54:32.452197 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:32.452159    5907 generic.go:356] "Generic (PLEG): container finished" podID="919c056f-1a01-4b8f-a6ee-0f49f79e64ad" containerID="373869832d6f1cf3614c31c80a4082c4944493c555fd16e476ee01ecba483a52" exitCode=0
Oct 30 19:54:32.452631 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:32.452248    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kcqxp" event={"ID":"919c056f-1a01-4b8f-a6ee-0f49f79e64ad","Type":"ContainerDied","Data":"373869832d6f1cf3614c31c80a4082c4944493c555fd16e476ee01ecba483a52"}
Oct 30 19:54:33.622719 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:33.622696    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kcqxp"
Oct 30 19:54:33.763520 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:33.763434    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/919c056f-1a01-4b8f-a6ee-0f49f79e64ad-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/919c056f-1a01-4b8f-a6ee-0f49f79e64ad-maas-api-xks-ca\") pod \"919c056f-1a01-4b8f-a6ee-0f49f79e64ad\" (UID: \"919c056f-1a01-4b8f-a6ee-0f49f79e64ad\") "
Oct 30 19:54:33.763711 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:33.763532    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/919c056f-1a01-4b8f-a6ee-0f49f79e64ad-kube-api-access-krtfk\" (UniqueName: \"kubernetes.io/projected/919c056f-1a01-4b8f-a6ee-0f49f79e64ad-kube-api-access-krtfk\") pod \"919c056f-1a01-4b8f-a6ee-0f49f79e64ad\" (UID: \"919c056f-1a01-4b8f-a6ee-0f49f79e64ad\") "
Oct 30 19:54:33.763711 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:33.763575    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/919c056f-1a01-4b8f-a6ee-0f49f79e64ad-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/919c056f-1a01-4b8f-a6ee-0f49f79e64ad-maas-api-openshift-ca\") pod \"919c056f-1a01-4b8f-a6ee-0f49f79e64ad\" (UID: \"919c056f-1a01-4b8f-a6ee-0f49f79e64ad\") "
Oct 30 19:54:33.764041 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:33.764017    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/919c056f-1a01-4b8f-a6ee-0f49f79e64ad-maas-api-openshift-ca" pod "919c056f-1a01-4b8f-a6ee-0f49f79e64ad" (UID: "919c056f-1a01-4b8f-a6ee-0f49f79e64ad"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:54:33.765711 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:33.765687    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/919c056f-1a01-4b8f-a6ee-0f49f79e64ad-kube-api-access-krtfk" pod "919c056f-1a01-4b8f-a6ee-0f49f79e64ad" (UID: "919c056f-1a01-4b8f-a6ee-0f49f79e64ad"). InnerVolumeSpecName "kube-api-access-krtfk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:54:33.765833 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:33.765717    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/919c056f-1a01-4b8f-a6ee-0f49f79e64ad-maas-api-xks-ca" pod "919c056f-1a01-4b8f-a6ee-0f49f79e64ad" (UID: "919c056f-1a01-4b8f-a6ee-0f49f79e64ad"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:54:33.865135 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:33.865097    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/919c056f-1a01-4b8f-a6ee-0f49f79e64ad-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:54:33.865135 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:33.865128    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krtfk\" (UniqueName: \"kubernetes.io/projected/919c056f-1a01-4b8f-a6ee-0f49f79e64ad-kube-api-access-krtfk\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:54:33.865135 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:33.865141    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/919c056f-1a01-4b8f-a6ee-0f49f79e64ad-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:54:34.464920 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:34.464891    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kcqxp"
Oct 30 19:54:34.464920 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:34.464911    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kcqxp" event={"ID":"919c056f-1a01-4b8f-a6ee-0f49f79e64ad","Type":"ContainerDied","Data":"b1165039eeae69b88f2abf1d5b58319675d4d440a628c6202df0325b27b2d0e0"}
Oct 30 19:54:34.465123 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:34.464957    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b1165039eeae69b88f2abf1d5b58319675d4d440a628c6202df0325b27b2d0e0"
Oct 30 19:54:34.613590 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:34.613555    5907 scope.go:122] "RemoveContainer" containerID="8687fbdc0eb1827e2854f1c4e917034b7e671a442938129d8540e5f0b256f209"
Oct 30 19:54:34.622815 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:34.622798    5907 scope.go:122] "RemoveContainer" containerID="5b5244c8150d9d0ad6d27e7d36057ab054193986d0cac7633e0ad426a3ee7ac6"
Oct 30 19:54:34.630976 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:34.630952    5907 scope.go:122] "RemoveContainer" containerID="ef4ff1c73b95baa47f3277815baafe4a52f32ad97f5eeeee37499e1e40d61fb7"
Oct 30 19:54:34.639018 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:34.639001    5907 scope.go:122] "RemoveContainer" containerID="7b2f2db1917cff6c20bd4af02b1c8882c193ef198065759b4ee69a0b8f4e0b28"
Oct 30 19:54:34.646730 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:34.646709    5907 scope.go:122] "RemoveContainer" containerID="a965ee70eba673c8d0e1d65930079ae9d53cc9e0f76b983aed4f78556ab1ab99"
Oct 30 19:54:40.763695 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:40.763577    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-zpswv"]
Oct 30 19:54:40.764216 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:40.764100    5907 cpu_manager.go:414] "containerMap: removing container" podUID="919c056f-1a01-4b8f-a6ee-0f49f79e64ad" containerName="revoke-keys"
Oct 30 19:54:40.764216 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:40.764112    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="919c056f-1a01-4b8f-a6ee-0f49f79e64ad" containerName="revoke-keys"
Oct 30 19:54:40.764216 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:40.764194    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="919c056f-1a01-4b8f-a6ee-0f49f79e64ad" containerName="revoke-keys"
Oct 30 19:54:40.770239 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:40.770211    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-zpswv"
Oct 30 19:54:40.786562 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:40.786528    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-zpswv"]
Oct 30 19:54:40.797224 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:40.797197    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:54:40.797444 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:40.797228    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:54:40.932901 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:40.932844    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d27dd847-6b5a-4645-a571-c5b7271fdf88-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5a29efb3-zpswv\" (UID: \"d27dd847-6b5a-4645-a571-c5b7271fdf88\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-zpswv"
Oct 30 19:54:40.932901 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:40.932906    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrwbh\" (UniqueName: \"kubernetes.io/projected/d27dd847-6b5a-4645-a571-c5b7271fdf88-kube-api-access-nrwbh\") pod \"maas-api-revoke-sub-5a29efb3-zpswv\" (UID: \"d27dd847-6b5a-4645-a571-c5b7271fdf88\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-zpswv"
Oct 30 19:54:40.933127 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:40.933037    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d27dd847-6b5a-4645-a571-c5b7271fdf88-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5a29efb3-zpswv\" (UID: \"d27dd847-6b5a-4645-a571-c5b7271fdf88\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-zpswv"
Oct 30 19:54:41.034720 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:41.034640    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d27dd847-6b5a-4645-a571-c5b7271fdf88-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5a29efb3-zpswv\" (UID: \"d27dd847-6b5a-4645-a571-c5b7271fdf88\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-zpswv"
Oct 30 19:54:41.035052 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:41.034745    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d27dd847-6b5a-4645-a571-c5b7271fdf88-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5a29efb3-zpswv\" (UID: \"d27dd847-6b5a-4645-a571-c5b7271fdf88\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-zpswv"
Oct 30 19:54:41.035052 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:41.034776    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrwbh\" (UniqueName: \"kubernetes.io/projected/d27dd847-6b5a-4645-a571-c5b7271fdf88-kube-api-access-nrwbh\") pod \"maas-api-revoke-sub-5a29efb3-zpswv\" (UID: \"d27dd847-6b5a-4645-a571-c5b7271fdf88\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-zpswv"
Oct 30 19:54:41.035404 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:41.035383    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d27dd847-6b5a-4645-a571-c5b7271fdf88-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5a29efb3-zpswv\" (UID: \"d27dd847-6b5a-4645-a571-c5b7271fdf88\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-zpswv"
Oct 30 19:54:41.037120 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:41.037102    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d27dd847-6b5a-4645-a571-c5b7271fdf88-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5a29efb3-zpswv\" (UID: \"d27dd847-6b5a-4645-a571-c5b7271fdf88\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-zpswv"
Oct 30 19:54:41.055490 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:41.055464    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrwbh\" (UniqueName: \"kubernetes.io/projected/d27dd847-6b5a-4645-a571-c5b7271fdf88-kube-api-access-nrwbh\") pod \"maas-api-revoke-sub-5a29efb3-zpswv\" (UID: \"d27dd847-6b5a-4645-a571-c5b7271fdf88\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-zpswv"
Oct 30 19:54:41.082924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:41.082890    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-zpswv"
Oct 30 19:54:41.385091 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:41.384997    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-zpswv"]
Oct 30 19:54:41.387614 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:54:41.387580    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd27dd847_6b5a_4645_a571_c5b7271fdf88.slice/crio-0be977cf1650c419b45e8be2e29d02d98ab40af7d54b856dc30f161e8ecfa245 WatchSource:0}: Error finding container 0be977cf1650c419b45e8be2e29d02d98ab40af7d54b856dc30f161e8ecfa245: Status 404 returned error can't find the container with id 0be977cf1650c419b45e8be2e29d02d98ab40af7d54b856dc30f161e8ecfa245
Oct 30 19:54:41.501773 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:41.501743    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-zpswv" event={"ID":"d27dd847-6b5a-4645-a571-c5b7271fdf88","Type":"ContainerStarted","Data":"0be977cf1650c419b45e8be2e29d02d98ab40af7d54b856dc30f161e8ecfa245"}
Oct 30 19:54:41.770386 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:41.770351    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:54:42.512019 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:42.511909    5907 generic.go:356] "Generic (PLEG): container finished" podID="d27dd847-6b5a-4645-a571-c5b7271fdf88" containerID="2832ffbb9a2bf7ad03c253d476f988297f81745120aa3a87e06d75cb1a3d1366" exitCode=0
Oct 30 19:54:42.512210 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:42.512013    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-zpswv" event={"ID":"d27dd847-6b5a-4645-a571-c5b7271fdf88","Type":"ContainerDied","Data":"2832ffbb9a2bf7ad03c253d476f988297f81745120aa3a87e06d75cb1a3d1366"}
Oct 30 19:54:44.521402 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:44.521363    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-zpswv"
Oct 30 19:54:44.555888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:44.555748    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d27dd847-6b5a-4645-a571-c5b7271fdf88-kube-api-access-nrwbh\" (UniqueName: \"kubernetes.io/projected/d27dd847-6b5a-4645-a571-c5b7271fdf88-kube-api-access-nrwbh\") pod \"d27dd847-6b5a-4645-a571-c5b7271fdf88\" (UID: \"d27dd847-6b5a-4645-a571-c5b7271fdf88\") "
Oct 30 19:54:44.555888 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:44.555809    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d27dd847-6b5a-4645-a571-c5b7271fdf88-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d27dd847-6b5a-4645-a571-c5b7271fdf88-maas-api-openshift-ca\") pod \"d27dd847-6b5a-4645-a571-c5b7271fdf88\" (UID: \"d27dd847-6b5a-4645-a571-c5b7271fdf88\") "
Oct 30 19:54:44.556175 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:44.555936    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d27dd847-6b5a-4645-a571-c5b7271fdf88-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d27dd847-6b5a-4645-a571-c5b7271fdf88-maas-api-xks-ca\") pod \"d27dd847-6b5a-4645-a571-c5b7271fdf88\" (UID: \"d27dd847-6b5a-4645-a571-c5b7271fdf88\") "
Oct 30 19:54:44.560885 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:44.560097    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d27dd847-6b5a-4645-a571-c5b7271fdf88-maas-api-openshift-ca" pod "d27dd847-6b5a-4645-a571-c5b7271fdf88" (UID: "d27dd847-6b5a-4645-a571-c5b7271fdf88"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:54:44.564816 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:44.564780    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-zpswv" event={"ID":"d27dd847-6b5a-4645-a571-c5b7271fdf88","Type":"ContainerDied","Data":"0be977cf1650c419b45e8be2e29d02d98ab40af7d54b856dc30f161e8ecfa245"}
Oct 30 19:54:44.565089 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:44.565072    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0be977cf1650c419b45e8be2e29d02d98ab40af7d54b856dc30f161e8ecfa245"
Oct 30 19:54:44.565294 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:44.565278    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-zpswv"
Oct 30 19:54:44.602267 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:44.602229    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d27dd847-6b5a-4645-a571-c5b7271fdf88-kube-api-access-nrwbh" pod "d27dd847-6b5a-4645-a571-c5b7271fdf88" (UID: "d27dd847-6b5a-4645-a571-c5b7271fdf88"). InnerVolumeSpecName "kube-api-access-nrwbh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:54:44.602496 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:44.602378    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d27dd847-6b5a-4645-a571-c5b7271fdf88-maas-api-xks-ca" pod "d27dd847-6b5a-4645-a571-c5b7271fdf88" (UID: "d27dd847-6b5a-4645-a571-c5b7271fdf88"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:54:44.679749 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:44.679651    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrwbh\" (UniqueName: \"kubernetes.io/projected/d27dd847-6b5a-4645-a571-c5b7271fdf88-kube-api-access-nrwbh\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:54:44.698288 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:44.698032    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d27dd847-6b5a-4645-a571-c5b7271fdf88-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:54:44.698288 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:44.698063    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d27dd847-6b5a-4645-a571-c5b7271fdf88-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:54:47.499740 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:54:47.499701    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:54:52.081521 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:52.081486    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:54:59.860540 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:59.860502    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pc86p"]
Oct 30 19:54:59.874086 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:59.874055    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pc86p"]
Oct 30 19:54:59.902431 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:59.902403    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jvlrj"]
Oct 30 19:54:59.902869 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:59.902846    5907 cpu_manager.go:414] "containerMap: removing container" podUID="d27dd847-6b5a-4645-a571-c5b7271fdf88" containerName="revoke-keys"
Oct 30 19:54:59.902919 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:59.902877    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d27dd847-6b5a-4645-a571-c5b7271fdf88" containerName="revoke-keys"
Oct 30 19:54:59.902963 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:59.902952    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="d27dd847-6b5a-4645-a571-c5b7271fdf88" containerName="revoke-keys"
Oct 30 19:54:59.906367 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:59.906342    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jvlrj"
Oct 30 19:54:59.923035 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:59.922999    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:54:59.923252 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:59.923233    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:54:59.929431 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:54:59.929397    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jvlrj"]
Oct 30 19:55:00.055209 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:00.055177    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9ns5\" (UniqueName: \"kubernetes.io/projected/68ed344e-86bf-4b00-8c9d-5298f3db5357-kube-api-access-l9ns5\") pod \"maas-api-revoke-sub-3d4156f1-jvlrj\" (UID: \"68ed344e-86bf-4b00-8c9d-5298f3db5357\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jvlrj"
Oct 30 19:55:00.055402 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:00.055216    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/68ed344e-86bf-4b00-8c9d-5298f3db5357-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-jvlrj\" (UID: \"68ed344e-86bf-4b00-8c9d-5298f3db5357\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jvlrj"
Oct 30 19:55:00.055452 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:00.055395    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/68ed344e-86bf-4b00-8c9d-5298f3db5357-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-jvlrj\" (UID: \"68ed344e-86bf-4b00-8c9d-5298f3db5357\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jvlrj"
Oct 30 19:55:00.156096 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:00.155996    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/68ed344e-86bf-4b00-8c9d-5298f3db5357-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-jvlrj\" (UID: \"68ed344e-86bf-4b00-8c9d-5298f3db5357\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jvlrj"
Oct 30 19:55:00.156096 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:00.156081    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9ns5\" (UniqueName: \"kubernetes.io/projected/68ed344e-86bf-4b00-8c9d-5298f3db5357-kube-api-access-l9ns5\") pod \"maas-api-revoke-sub-3d4156f1-jvlrj\" (UID: \"68ed344e-86bf-4b00-8c9d-5298f3db5357\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jvlrj"
Oct 30 19:55:00.156358 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:00.156102    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/68ed344e-86bf-4b00-8c9d-5298f3db5357-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-jvlrj\" (UID: \"68ed344e-86bf-4b00-8c9d-5298f3db5357\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jvlrj"
Oct 30 19:55:00.156820 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:00.156799    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/68ed344e-86bf-4b00-8c9d-5298f3db5357-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-jvlrj\" (UID: \"68ed344e-86bf-4b00-8c9d-5298f3db5357\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jvlrj"
Oct 30 19:55:00.158594 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:00.158568    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/68ed344e-86bf-4b00-8c9d-5298f3db5357-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-jvlrj\" (UID: \"68ed344e-86bf-4b00-8c9d-5298f3db5357\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jvlrj"
Oct 30 19:55:00.177883 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:00.177834    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9ns5\" (UniqueName: \"kubernetes.io/projected/68ed344e-86bf-4b00-8c9d-5298f3db5357-kube-api-access-l9ns5\") pod \"maas-api-revoke-sub-3d4156f1-jvlrj\" (UID: \"68ed344e-86bf-4b00-8c9d-5298f3db5357\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jvlrj"
Oct 30 19:55:00.220987 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:00.220963    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jvlrj"
Oct 30 19:55:00.291902 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:00.291807    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="019e0695-5b4c-4efe-8360-a128b10e60e0" path="/var/lib/kubelet/pods/019e0695-5b4c-4efe-8360-a128b10e60e0/volumes"
Oct 30 19:55:00.524219 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:00.524147    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jvlrj"]
Oct 30 19:55:00.528472 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:55:00.528438    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68ed344e_86bf_4b00_8c9d_5298f3db5357.slice/crio-c10e882885c70877803bedb73cd3e1711f64fa229fd8dc7d2ba78635e014d908 WatchSource:0}: Error finding container c10e882885c70877803bedb73cd3e1711f64fa229fd8dc7d2ba78635e014d908: Status 404 returned error can't find the container with id c10e882885c70877803bedb73cd3e1711f64fa229fd8dc7d2ba78635e014d908
Oct 30 19:55:00.652588 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:00.652552    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jvlrj" event={"ID":"68ed344e-86bf-4b00-8c9d-5298f3db5357","Type":"ContainerStarted","Data":"c10e882885c70877803bedb73cd3e1711f64fa229fd8dc7d2ba78635e014d908"}
Oct 30 19:55:01.028580 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:01.028537    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-lmn6m"]
Oct 30 19:55:01.041724 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:01.041693    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-lmn6m"]
Oct 30 19:55:01.660898 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:01.660782    5907 generic.go:356] "Generic (PLEG): container finished" podID="68ed344e-86bf-4b00-8c9d-5298f3db5357" containerID="dd74af0fcd1ffd1dea6c17995b8169d9e798a1e86e2b0b3def8598440f5cbbca" exitCode=0
Oct 30 19:55:01.660898 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:01.660878    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jvlrj" event={"ID":"68ed344e-86bf-4b00-8c9d-5298f3db5357","Type":"ContainerDied","Data":"dd74af0fcd1ffd1dea6c17995b8169d9e798a1e86e2b0b3def8598440f5cbbca"}
Oct 30 19:55:02.287443 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:02.287408    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="56409091-d2f5-43ab-9412-d95605c5e3b4" path="/var/lib/kubelet/pods/56409091-d2f5-43ab-9412-d95605c5e3b4/volumes"
Oct 30 19:55:02.815797 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:02.815774    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jvlrj"
Oct 30 19:55:02.989668 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:02.989587    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/68ed344e-86bf-4b00-8c9d-5298f3db5357-kube-api-access-l9ns5\" (UniqueName: \"kubernetes.io/projected/68ed344e-86bf-4b00-8c9d-5298f3db5357-kube-api-access-l9ns5\") pod \"68ed344e-86bf-4b00-8c9d-5298f3db5357\" (UID: \"68ed344e-86bf-4b00-8c9d-5298f3db5357\") "
Oct 30 19:55:02.989668 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:02.989663    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/68ed344e-86bf-4b00-8c9d-5298f3db5357-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/68ed344e-86bf-4b00-8c9d-5298f3db5357-maas-api-xks-ca\") pod \"68ed344e-86bf-4b00-8c9d-5298f3db5357\" (UID: \"68ed344e-86bf-4b00-8c9d-5298f3db5357\") "
Oct 30 19:55:02.989920 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:02.989756    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/68ed344e-86bf-4b00-8c9d-5298f3db5357-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/68ed344e-86bf-4b00-8c9d-5298f3db5357-maas-api-openshift-ca\") pod \"68ed344e-86bf-4b00-8c9d-5298f3db5357\" (UID: \"68ed344e-86bf-4b00-8c9d-5298f3db5357\") "
Oct 30 19:55:02.990171 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:02.990149    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68ed344e-86bf-4b00-8c9d-5298f3db5357-maas-api-openshift-ca" pod "68ed344e-86bf-4b00-8c9d-5298f3db5357" (UID: "68ed344e-86bf-4b00-8c9d-5298f3db5357"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:55:02.990249 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:02.990228    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/68ed344e-86bf-4b00-8c9d-5298f3db5357-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:55:02.991897 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:02.991854    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68ed344e-86bf-4b00-8c9d-5298f3db5357-maas-api-xks-ca" pod "68ed344e-86bf-4b00-8c9d-5298f3db5357" (UID: "68ed344e-86bf-4b00-8c9d-5298f3db5357"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:55:02.992006 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:02.991985    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68ed344e-86bf-4b00-8c9d-5298f3db5357-kube-api-access-l9ns5" pod "68ed344e-86bf-4b00-8c9d-5298f3db5357" (UID: "68ed344e-86bf-4b00-8c9d-5298f3db5357"). InnerVolumeSpecName "kube-api-access-l9ns5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:55:03.091343 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:03.091314    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l9ns5\" (UniqueName: \"kubernetes.io/projected/68ed344e-86bf-4b00-8c9d-5298f3db5357-kube-api-access-l9ns5\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:55:03.091343 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:03.091342    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/68ed344e-86bf-4b00-8c9d-5298f3db5357-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:55:03.674690 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:03.674652    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jvlrj" event={"ID":"68ed344e-86bf-4b00-8c9d-5298f3db5357","Type":"ContainerDied","Data":"c10e882885c70877803bedb73cd3e1711f64fa229fd8dc7d2ba78635e014d908"}
Oct 30 19:55:03.674690 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:03.674671    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jvlrj"
Oct 30 19:55:03.675219 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:03.674709    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c10e882885c70877803bedb73cd3e1711f64fa229fd8dc7d2ba78635e014d908"
Oct 30 19:55:17.499178 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:55:17.499141    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:55:22.564418 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:22.564385    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:55:34.682935 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:34.682900    5907 scope.go:122] "RemoveContainer" containerID="09107fde4b9e170b8c3ff6c6828f8d8596ed2a5fa8dd14620be3c06bb3e456f8"
Oct 30 19:55:34.692448 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:34.692427    5907 scope.go:122] "RemoveContainer" containerID="53014dabaab60ab459b86dedef3147560d5696e6f53bf67a2dd0c0b9e12138ed"
Oct 30 19:55:35.877980 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:35.877943    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:55:47.500308 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:55:47.500263    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:55:49.304398 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:55:49.304362    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:56:02.026569 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:02.026526    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4vr4k"]
Oct 30 19:56:02.039702 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:02.039667    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-4vr4k"]
Oct 30 19:56:02.313962 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:02.311532    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0" path="/var/lib/kubelet/pods/4de21dd4-2ce4-4ead-bfc9-d17bf7f4a7c0/volumes"
Oct 30 19:56:03.067455 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:03.067416    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a45e4b2b-k7h6z"]
Oct 30 19:56:03.067915 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:03.067900    5907 cpu_manager.go:414] "containerMap: removing container" podUID="68ed344e-86bf-4b00-8c9d-5298f3db5357" containerName="revoke-keys"
Oct 30 19:56:03.067972 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:03.067916    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="68ed344e-86bf-4b00-8c9d-5298f3db5357" containerName="revoke-keys"
Oct 30 19:56:03.068027 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:03.068017    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="68ed344e-86bf-4b00-8c9d-5298f3db5357" containerName="revoke-keys"
Oct 30 19:56:03.071584 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:03.071568    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a45e4b2b-k7h6z"
Oct 30 19:56:03.089387 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:03.089366    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:56:03.089600 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:03.089581    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:56:03.092690 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:03.092656    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a45e4b2b-k7h6z"]
Oct 30 19:56:03.178676 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:03.178647    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1ee0f668-16b6-43ca-b77d-c16bf3cd6347-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a45e4b2b-k7h6z\" (UID: \"1ee0f668-16b6-43ca-b77d-c16bf3cd6347\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a45e4b2b-k7h6z"
Oct 30 19:56:03.178851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:03.178695    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1ee0f668-16b6-43ca-b77d-c16bf3cd6347-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a45e4b2b-k7h6z\" (UID: \"1ee0f668-16b6-43ca-b77d-c16bf3cd6347\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a45e4b2b-k7h6z"
Oct 30 19:56:03.178851 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:03.178817    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frxr8\" (UniqueName: \"kubernetes.io/projected/1ee0f668-16b6-43ca-b77d-c16bf3cd6347-kube-api-access-frxr8\") pod \"maas-api-revoke-sub-a45e4b2b-k7h6z\" (UID: \"1ee0f668-16b6-43ca-b77d-c16bf3cd6347\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a45e4b2b-k7h6z"
Oct 30 19:56:03.279876 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:03.279821    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1ee0f668-16b6-43ca-b77d-c16bf3cd6347-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a45e4b2b-k7h6z\" (UID: \"1ee0f668-16b6-43ca-b77d-c16bf3cd6347\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a45e4b2b-k7h6z"
Oct 30 19:56:03.280061 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:03.279960    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frxr8\" (UniqueName: \"kubernetes.io/projected/1ee0f668-16b6-43ca-b77d-c16bf3cd6347-kube-api-access-frxr8\") pod \"maas-api-revoke-sub-a45e4b2b-k7h6z\" (UID: \"1ee0f668-16b6-43ca-b77d-c16bf3cd6347\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a45e4b2b-k7h6z"
Oct 30 19:56:03.280168 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:03.280071    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1ee0f668-16b6-43ca-b77d-c16bf3cd6347-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a45e4b2b-k7h6z\" (UID: \"1ee0f668-16b6-43ca-b77d-c16bf3cd6347\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a45e4b2b-k7h6z"
Oct 30 19:56:03.280626 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:03.280602    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1ee0f668-16b6-43ca-b77d-c16bf3cd6347-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a45e4b2b-k7h6z\" (UID: \"1ee0f668-16b6-43ca-b77d-c16bf3cd6347\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a45e4b2b-k7h6z"
Oct 30 19:56:03.282561 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:03.282543    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1ee0f668-16b6-43ca-b77d-c16bf3cd6347-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a45e4b2b-k7h6z\" (UID: \"1ee0f668-16b6-43ca-b77d-c16bf3cd6347\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a45e4b2b-k7h6z"
Oct 30 19:56:03.300512 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:03.300480    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frxr8\" (UniqueName: \"kubernetes.io/projected/1ee0f668-16b6-43ca-b77d-c16bf3cd6347-kube-api-access-frxr8\") pod \"maas-api-revoke-sub-a45e4b2b-k7h6z\" (UID: \"1ee0f668-16b6-43ca-b77d-c16bf3cd6347\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a45e4b2b-k7h6z"
Oct 30 19:56:03.384924 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:03.384904    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a45e4b2b-k7h6z"
Oct 30 19:56:03.687343 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:03.687316    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a45e4b2b-k7h6z"]
Oct 30 19:56:03.689440 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:56:03.689410    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ee0f668_16b6_43ca_b77d_c16bf3cd6347.slice/crio-ba7fde836d3190e0231e9e72d2e517859cd903bd1fbb710ff2f3ba4353b3e62f WatchSource:0}: Error finding container ba7fde836d3190e0231e9e72d2e517859cd903bd1fbb710ff2f3ba4353b3e62f: Status 404 returned error can't find the container with id ba7fde836d3190e0231e9e72d2e517859cd903bd1fbb710ff2f3ba4353b3e62f
Oct 30 19:56:04.013634 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:04.013541    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a45e4b2b-k7h6z" event={"ID":"1ee0f668-16b6-43ca-b77d-c16bf3cd6347","Type":"ContainerStarted","Data":"ba7fde836d3190e0231e9e72d2e517859cd903bd1fbb710ff2f3ba4353b3e62f"}
Oct 30 19:56:05.020162 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:05.020067    5907 generic.go:356] "Generic (PLEG): container finished" podID="1ee0f668-16b6-43ca-b77d-c16bf3cd6347" containerID="19b92d6b42c4989eedb36ae6911e462bae1ff5f83de90e2500e91f783fae0c52" exitCode=0
Oct 30 19:56:05.020566 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:05.020161    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a45e4b2b-k7h6z" event={"ID":"1ee0f668-16b6-43ca-b77d-c16bf3cd6347","Type":"ContainerDied","Data":"19b92d6b42c4989eedb36ae6911e462bae1ff5f83de90e2500e91f783fae0c52"}
Oct 30 19:56:06.164455 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:06.164425    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a45e4b2b-k7h6z"
Oct 30 19:56:06.318709 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:06.318634    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1ee0f668-16b6-43ca-b77d-c16bf3cd6347-kube-api-access-frxr8\" (UniqueName: \"kubernetes.io/projected/1ee0f668-16b6-43ca-b77d-c16bf3cd6347-kube-api-access-frxr8\") pod \"1ee0f668-16b6-43ca-b77d-c16bf3cd6347\" (UID: \"1ee0f668-16b6-43ca-b77d-c16bf3cd6347\") "
Oct 30 19:56:06.318709 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:06.318665    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1ee0f668-16b6-43ca-b77d-c16bf3cd6347-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1ee0f668-16b6-43ca-b77d-c16bf3cd6347-maas-api-openshift-ca\") pod \"1ee0f668-16b6-43ca-b77d-c16bf3cd6347\" (UID: \"1ee0f668-16b6-43ca-b77d-c16bf3cd6347\") "
Oct 30 19:56:06.318928 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:06.318821    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1ee0f668-16b6-43ca-b77d-c16bf3cd6347-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1ee0f668-16b6-43ca-b77d-c16bf3cd6347-maas-api-xks-ca\") pod \"1ee0f668-16b6-43ca-b77d-c16bf3cd6347\" (UID: \"1ee0f668-16b6-43ca-b77d-c16bf3cd6347\") "
Oct 30 19:56:06.319076 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:06.319054    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ee0f668-16b6-43ca-b77d-c16bf3cd6347-maas-api-openshift-ca" pod "1ee0f668-16b6-43ca-b77d-c16bf3cd6347" (UID: "1ee0f668-16b6-43ca-b77d-c16bf3cd6347"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:56:06.320932 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:06.320904    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ee0f668-16b6-43ca-b77d-c16bf3cd6347-maas-api-xks-ca" pod "1ee0f668-16b6-43ca-b77d-c16bf3cd6347" (UID: "1ee0f668-16b6-43ca-b77d-c16bf3cd6347"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:56:06.321060 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:06.320976    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ee0f668-16b6-43ca-b77d-c16bf3cd6347-kube-api-access-frxr8" pod "1ee0f668-16b6-43ca-b77d-c16bf3cd6347" (UID: "1ee0f668-16b6-43ca-b77d-c16bf3cd6347"). InnerVolumeSpecName "kube-api-access-frxr8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:56:06.420383 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:06.420350    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1ee0f668-16b6-43ca-b77d-c16bf3cd6347-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:56:06.420383 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:06.420383    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frxr8\" (UniqueName: \"kubernetes.io/projected/1ee0f668-16b6-43ca-b77d-c16bf3cd6347-kube-api-access-frxr8\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:56:06.420573 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:06.420401    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1ee0f668-16b6-43ca-b77d-c16bf3cd6347-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:56:07.032508 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:07.032479    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a45e4b2b-k7h6z"
Oct 30 19:56:07.032701 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:07.032520    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a45e4b2b-k7h6z" event={"ID":"1ee0f668-16b6-43ca-b77d-c16bf3cd6347","Type":"ContainerDied","Data":"ba7fde836d3190e0231e9e72d2e517859cd903bd1fbb710ff2f3ba4353b3e62f"}
Oct 30 19:56:07.032701 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:07.032566    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ba7fde836d3190e0231e9e72d2e517859cd903bd1fbb710ff2f3ba4353b3e62f"
Oct 30 19:56:14.436670 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:14.436632    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-1f3297-llvc4"]
Oct 30 19:56:14.437135 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:14.437120    5907 cpu_manager.go:414] "containerMap: removing container" podUID="1ee0f668-16b6-43ca-b77d-c16bf3cd6347" containerName="revoke-keys"
Oct 30 19:56:14.437135 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:14.437135    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1ee0f668-16b6-43ca-b77d-c16bf3cd6347" containerName="revoke-keys"
Oct 30 19:56:14.437219 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:14.437213    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="1ee0f668-16b6-43ca-b77d-c16bf3cd6347" containerName="revoke-keys"
Oct 30 19:56:14.440659 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:14.440642    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-1f3297-llvc4"
Oct 30 19:56:14.460982 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:14.460951    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-1f3297-llvc4"]
Oct 30 19:56:14.471595 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:14.471570    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:56:14.471769 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:14.471607    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:56:14.516044 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:14.516005    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/099497eb-5750-4680-9313-12c6f5a14b4b-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-1f3297-llvc4\" (UID: \"099497eb-5750-4680-9313-12c6f5a14b4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-1f3297-llvc4"
Oct 30 19:56:14.516345 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:14.516312    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkmrt\" (UniqueName: \"kubernetes.io/projected/099497eb-5750-4680-9313-12c6f5a14b4b-kube-api-access-dkmrt\") pod \"maas-api-revoke-keys-e2e-worker-single-main-1f3297-llvc4\" (UID: \"099497eb-5750-4680-9313-12c6f5a14b4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-1f3297-llvc4"
Oct 30 19:56:14.516492 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:14.516406    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/099497eb-5750-4680-9313-12c6f5a14b4b-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-1f3297-llvc4\" (UID: \"099497eb-5750-4680-9313-12c6f5a14b4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-1f3297-llvc4"
Oct 30 19:56:14.617706 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:14.617666    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/099497eb-5750-4680-9313-12c6f5a14b4b-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-1f3297-llvc4\" (UID: \"099497eb-5750-4680-9313-12c6f5a14b4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-1f3297-llvc4"
Oct 30 19:56:14.617948 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:14.617718    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkmrt\" (UniqueName: \"kubernetes.io/projected/099497eb-5750-4680-9313-12c6f5a14b4b-kube-api-access-dkmrt\") pod \"maas-api-revoke-keys-e2e-worker-single-main-1f3297-llvc4\" (UID: \"099497eb-5750-4680-9313-12c6f5a14b4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-1f3297-llvc4"
Oct 30 19:56:14.617948 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:14.617780    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/099497eb-5750-4680-9313-12c6f5a14b4b-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-1f3297-llvc4\" (UID: \"099497eb-5750-4680-9313-12c6f5a14b4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-1f3297-llvc4"
Oct 30 19:56:14.618539 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:14.618519    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/099497eb-5750-4680-9313-12c6f5a14b4b-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-1f3297-llvc4\" (UID: \"099497eb-5750-4680-9313-12c6f5a14b4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-1f3297-llvc4"
Oct 30 19:56:14.620263 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:14.620238    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/099497eb-5750-4680-9313-12c6f5a14b4b-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-1f3297-llvc4\" (UID: \"099497eb-5750-4680-9313-12c6f5a14b4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-1f3297-llvc4"
Oct 30 19:56:14.641153 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:14.641128    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkmrt\" (UniqueName: \"kubernetes.io/projected/099497eb-5750-4680-9313-12c6f5a14b4b-kube-api-access-dkmrt\") pod \"maas-api-revoke-keys-e2e-worker-single-main-1f3297-llvc4\" (UID: \"099497eb-5750-4680-9313-12c6f5a14b4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-1f3297-llvc4"
Oct 30 19:56:14.753684 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:14.753598    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-1f3297-llvc4"
Oct 30 19:56:15.057784 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:15.057754    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-1f3297-llvc4"]
Oct 30 19:56:15.059591 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:56:15.059562    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod099497eb_5750_4680_9313_12c6f5a14b4b.slice/crio-cd17e7ab8835e72ddd75aca6fd53891ced073463c93132c1f1739ad73385059d WatchSource:0}: Error finding container cd17e7ab8835e72ddd75aca6fd53891ced073463c93132c1f1739ad73385059d: Status 404 returned error can't find the container with id cd17e7ab8835e72ddd75aca6fd53891ced073463c93132c1f1739ad73385059d
Oct 30 19:56:15.081696 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:15.081662    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-1f3297-llvc4" event={"ID":"099497eb-5750-4680-9313-12c6f5a14b4b","Type":"ContainerStarted","Data":"cd17e7ab8835e72ddd75aca6fd53891ced073463c93132c1f1739ad73385059d"}
Oct 30 19:56:15.613242 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:15.613209    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:56:16.089151 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:16.089051    5907 generic.go:356] "Generic (PLEG): container finished" podID="099497eb-5750-4680-9313-12c6f5a14b4b" containerID="57204452be7cd38d9f03c8dea2e9facdf562a2e4140e44043c685c5ff4df67b6" exitCode=0
Oct 30 19:56:16.089151 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:16.089112    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-1f3297-llvc4" event={"ID":"099497eb-5750-4680-9313-12c6f5a14b4b","Type":"ContainerDied","Data":"57204452be7cd38d9f03c8dea2e9facdf562a2e4140e44043c685c5ff4df67b6"}
Oct 30 19:56:17.234730 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:17.234702    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-1f3297-llvc4"
Oct 30 19:56:17.346875 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:17.346774    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/099497eb-5750-4680-9313-12c6f5a14b4b-kube-api-access-dkmrt\" (UniqueName: \"kubernetes.io/projected/099497eb-5750-4680-9313-12c6f5a14b4b-kube-api-access-dkmrt\") pod \"099497eb-5750-4680-9313-12c6f5a14b4b\" (UID: \"099497eb-5750-4680-9313-12c6f5a14b4b\") "
Oct 30 19:56:17.347037 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:17.346919    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/099497eb-5750-4680-9313-12c6f5a14b4b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/099497eb-5750-4680-9313-12c6f5a14b4b-maas-api-xks-ca\") pod \"099497eb-5750-4680-9313-12c6f5a14b4b\" (UID: \"099497eb-5750-4680-9313-12c6f5a14b4b\") "
Oct 30 19:56:17.347037 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:17.346955    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/099497eb-5750-4680-9313-12c6f5a14b4b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/099497eb-5750-4680-9313-12c6f5a14b4b-maas-api-openshift-ca\") pod \"099497eb-5750-4680-9313-12c6f5a14b4b\" (UID: \"099497eb-5750-4680-9313-12c6f5a14b4b\") "
Oct 30 19:56:17.347352 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:17.347325    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/099497eb-5750-4680-9313-12c6f5a14b4b-maas-api-openshift-ca" pod "099497eb-5750-4680-9313-12c6f5a14b4b" (UID: "099497eb-5750-4680-9313-12c6f5a14b4b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:56:17.349116 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:17.349098    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/099497eb-5750-4680-9313-12c6f5a14b4b-maas-api-xks-ca" pod "099497eb-5750-4680-9313-12c6f5a14b4b" (UID: "099497eb-5750-4680-9313-12c6f5a14b4b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:56:17.349199 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:17.349184    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/099497eb-5750-4680-9313-12c6f5a14b4b-kube-api-access-dkmrt" pod "099497eb-5750-4680-9313-12c6f5a14b4b" (UID: "099497eb-5750-4680-9313-12c6f5a14b4b"). InnerVolumeSpecName "kube-api-access-dkmrt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:56:17.448724 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:17.448691    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/099497eb-5750-4680-9313-12c6f5a14b4b-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:56:17.448724 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:17.448720    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/099497eb-5750-4680-9313-12c6f5a14b4b-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:56:17.448724 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:17.448730    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dkmrt\" (UniqueName: \"kubernetes.io/projected/099497eb-5750-4680-9313-12c6f5a14b4b-kube-api-access-dkmrt\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:56:17.500949 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:56:17.500910    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:56:18.101498 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:18.101464    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-1f3297-llvc4"
Oct 30 19:56:18.101695 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:18.101503    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-1f3297-llvc4" event={"ID":"099497eb-5750-4680-9313-12c6f5a14b4b","Type":"ContainerDied","Data":"cd17e7ab8835e72ddd75aca6fd53891ced073463c93132c1f1739ad73385059d"}
Oct 30 19:56:18.101695 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:18.101551    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cd17e7ab8835e72ddd75aca6fd53891ced073463c93132c1f1739ad73385059d"
Oct 30 19:56:25.038983 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:25.038942    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5fsng"]
Oct 30 19:56:25.051726 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:25.051697    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-5fsng"]
Oct 30 19:56:26.286152 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:26.286122    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7eee3935-dacf-433c-b667-ef978af84873" path="/var/lib/kubelet/pods/7eee3935-dacf-433c-b667-ef978af84873/volumes"
Oct 30 19:56:27.522627 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:27.522592    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-1f3297-llvc4"]
Oct 30 19:56:27.535912 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:27.535883    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-1f3297-llvc4"]
Oct 30 19:56:27.774266 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:27.774145    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:56:28.286882 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:28.286823    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="099497eb-5750-4680-9313-12c6f5a14b4b" path="/var/lib/kubelet/pods/099497eb-5750-4680-9313-12c6f5a14b4b/volumes"
Oct 30 19:56:34.721323 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:34.721284    5907 scope.go:122] "RemoveContainer" containerID="f462dcc69551f0c49096736df39cfba06acfc583855f8ca0f518bd9e2d554bd5"
Oct 30 19:56:35.467722 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:35.467684    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jvlrj"]
Oct 30 19:56:35.479705 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:35.479677    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jvlrj"]
Oct 30 19:56:35.511641 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:35.511617    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-smhtk"]
Oct 30 19:56:35.512075 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:35.512062    5907 cpu_manager.go:414] "containerMap: removing container" podUID="099497eb-5750-4680-9313-12c6f5a14b4b" containerName="revoke-keys"
Oct 30 19:56:35.512128 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:35.512077    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="099497eb-5750-4680-9313-12c6f5a14b4b" containerName="revoke-keys"
Oct 30 19:56:35.512165 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:35.512158    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="099497eb-5750-4680-9313-12c6f5a14b4b" containerName="revoke-keys"
Oct 30 19:56:35.517073 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:35.517052    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-smhtk"
Oct 30 19:56:35.535825 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:35.535802    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-smhtk"]
Oct 30 19:56:35.536343 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:35.536323    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:56:35.536498 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:35.536391    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:56:35.658114 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:35.658078    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e12a469d-c556-4761-a182-8c5498d7b180-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-smhtk\" (UID: \"e12a469d-c556-4761-a182-8c5498d7b180\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-smhtk"
Oct 30 19:56:35.658337 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:35.658125    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e12a469d-c556-4761-a182-8c5498d7b180-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-smhtk\" (UID: \"e12a469d-c556-4761-a182-8c5498d7b180\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-smhtk"
Oct 30 19:56:35.658337 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:35.658202    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdvjl\" (UniqueName: \"kubernetes.io/projected/e12a469d-c556-4761-a182-8c5498d7b180-kube-api-access-pdvjl\") pod \"maas-api-revoke-sub-3d4156f1-smhtk\" (UID: \"e12a469d-c556-4761-a182-8c5498d7b180\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-smhtk"
Oct 30 19:56:35.759554 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:35.759476    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e12a469d-c556-4761-a182-8c5498d7b180-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-smhtk\" (UID: \"e12a469d-c556-4761-a182-8c5498d7b180\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-smhtk"
Oct 30 19:56:35.759554 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:35.759518    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e12a469d-c556-4761-a182-8c5498d7b180-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-smhtk\" (UID: \"e12a469d-c556-4761-a182-8c5498d7b180\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-smhtk"
Oct 30 19:56:35.759554 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:35.759546    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdvjl\" (UniqueName: \"kubernetes.io/projected/e12a469d-c556-4761-a182-8c5498d7b180-kube-api-access-pdvjl\") pod \"maas-api-revoke-sub-3d4156f1-smhtk\" (UID: \"e12a469d-c556-4761-a182-8c5498d7b180\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-smhtk"
Oct 30 19:56:35.760178 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:35.760150    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e12a469d-c556-4761-a182-8c5498d7b180-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-smhtk\" (UID: \"e12a469d-c556-4761-a182-8c5498d7b180\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-smhtk"
Oct 30 19:56:35.762064 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:35.762040    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e12a469d-c556-4761-a182-8c5498d7b180-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-smhtk\" (UID: \"e12a469d-c556-4761-a182-8c5498d7b180\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-smhtk"
Oct 30 19:56:35.780463 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:35.780430    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdvjl\" (UniqueName: \"kubernetes.io/projected/e12a469d-c556-4761-a182-8c5498d7b180-kube-api-access-pdvjl\") pod \"maas-api-revoke-sub-3d4156f1-smhtk\" (UID: \"e12a469d-c556-4761-a182-8c5498d7b180\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-smhtk"
Oct 30 19:56:35.830164 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:35.830138    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-smhtk"
Oct 30 19:56:36.034370 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:36.034281    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-zhs4l"]
Oct 30 19:56:36.047326 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:36.047291    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-zhs4l"]
Oct 30 19:56:36.132473 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:36.132448    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-smhtk"]
Oct 30 19:56:36.134427 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:56:36.134399    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode12a469d_c556_4761_a182_8c5498d7b180.slice/crio-dc3f0b1d2f02675032ba531d33a4f1aa12c8ca94e4c330a5ce3330f561104920 WatchSource:0}: Error finding container dc3f0b1d2f02675032ba531d33a4f1aa12c8ca94e4c330a5ce3330f561104920: Status 404 returned error can't find the container with id dc3f0b1d2f02675032ba531d33a4f1aa12c8ca94e4c330a5ce3330f561104920
Oct 30 19:56:36.209271 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:36.209246    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-smhtk" event={"ID":"e12a469d-c556-4761-a182-8c5498d7b180","Type":"ContainerStarted","Data":"dc3f0b1d2f02675032ba531d33a4f1aa12c8ca94e4c330a5ce3330f561104920"}
Oct 30 19:56:36.286565 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:36.286480    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="31af6090-882d-4de8-9a4d-4aae8239e8e5" path="/var/lib/kubelet/pods/31af6090-882d-4de8-9a4d-4aae8239e8e5/volumes"
Oct 30 19:56:36.287041 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:36.287020    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="68ed344e-86bf-4b00-8c9d-5298f3db5357" path="/var/lib/kubelet/pods/68ed344e-86bf-4b00-8c9d-5298f3db5357/volumes"
Oct 30 19:56:37.216599 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:37.216510    5907 generic.go:356] "Generic (PLEG): container finished" podID="e12a469d-c556-4761-a182-8c5498d7b180" containerID="256207638a28f7f9cb4e524d3e065efd11e7e40bc2d4d8e904cd100a30d30f2c" exitCode=0
Oct 30 19:56:37.217010 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:37.216604    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-smhtk" event={"ID":"e12a469d-c556-4761-a182-8c5498d7b180","Type":"ContainerDied","Data":"256207638a28f7f9cb4e524d3e065efd11e7e40bc2d4d8e904cd100a30d30f2c"}
Oct 30 19:56:38.372360 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:38.372334    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-smhtk"
Oct 30 19:56:38.394423 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:38.394397    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e12a469d-c556-4761-a182-8c5498d7b180-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e12a469d-c556-4761-a182-8c5498d7b180-maas-api-openshift-ca\") pod \"e12a469d-c556-4761-a182-8c5498d7b180\" (UID: \"e12a469d-c556-4761-a182-8c5498d7b180\") "
Oct 30 19:56:38.394518 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:38.394449    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e12a469d-c556-4761-a182-8c5498d7b180-kube-api-access-pdvjl\" (UniqueName: \"kubernetes.io/projected/e12a469d-c556-4761-a182-8c5498d7b180-kube-api-access-pdvjl\") pod \"e12a469d-c556-4761-a182-8c5498d7b180\" (UID: \"e12a469d-c556-4761-a182-8c5498d7b180\") "
Oct 30 19:56:38.394518 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:38.394506    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e12a469d-c556-4761-a182-8c5498d7b180-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e12a469d-c556-4761-a182-8c5498d7b180-maas-api-xks-ca\") pod \"e12a469d-c556-4761-a182-8c5498d7b180\" (UID: \"e12a469d-c556-4761-a182-8c5498d7b180\") "
Oct 30 19:56:38.394776 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:38.394753    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e12a469d-c556-4761-a182-8c5498d7b180-maas-api-openshift-ca" pod "e12a469d-c556-4761-a182-8c5498d7b180" (UID: "e12a469d-c556-4761-a182-8c5498d7b180"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:56:38.394919 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:38.394853    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e12a469d-c556-4761-a182-8c5498d7b180-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:56:38.396693 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:38.396665    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e12a469d-c556-4761-a182-8c5498d7b180-kube-api-access-pdvjl" pod "e12a469d-c556-4761-a182-8c5498d7b180" (UID: "e12a469d-c556-4761-a182-8c5498d7b180"). InnerVolumeSpecName "kube-api-access-pdvjl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:56:38.396802 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:38.396742    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e12a469d-c556-4761-a182-8c5498d7b180-maas-api-xks-ca" pod "e12a469d-c556-4761-a182-8c5498d7b180" (UID: "e12a469d-c556-4761-a182-8c5498d7b180"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:56:38.495549 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:38.495452    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e12a469d-c556-4761-a182-8c5498d7b180-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:56:38.495549 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:38.495486    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pdvjl\" (UniqueName: \"kubernetes.io/projected/e12a469d-c556-4761-a182-8c5498d7b180-kube-api-access-pdvjl\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:56:39.228787 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:39.228753    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-smhtk"
Oct 30 19:56:39.229001 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:39.228797    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-smhtk" event={"ID":"e12a469d-c556-4761-a182-8c5498d7b180","Type":"ContainerDied","Data":"dc3f0b1d2f02675032ba531d33a4f1aa12c8ca94e4c330a5ce3330f561104920"}
Oct 30 19:56:39.229001 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:56:39.228849    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dc3f0b1d2f02675032ba531d33a4f1aa12c8ca94e4c330a5ce3330f561104920"
Oct 30 19:56:47.500487 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:56:47.500451    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:57:04.028455 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:57:04.028422    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-drrmq"]
Oct 30 19:57:04.041616 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:57:04.041586    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-drrmq"]
Oct 30 19:57:04.287155 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:57:04.287076    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="de2eb2ff-8abf-479e-9297-92174f893b1a" path="/var/lib/kubelet/pods/de2eb2ff-8abf-479e-9297-92174f893b1a/volumes"
Oct 30 19:57:15.030137 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:57:15.030099    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-7r7bc"]
Oct 30 19:57:15.043172 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:57:15.043140    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-7r7bc"]
Oct 30 19:57:16.286511 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:57:16.286474    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b8916631-dd19-4062-b5e6-1d68976378dd" path="/var/lib/kubelet/pods/b8916631-dd19-4062-b5e6-1d68976378dd/volumes"
Oct 30 19:57:17.500042 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:57:17.500002    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:57:34.744000 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:57:34.743969    5907 scope.go:122] "RemoveContainer" containerID="104350eb38c20f43703a9114f7e940ca96ba94ca1d3e561b4b387cd9b5aafc66"
Oct 30 19:57:34.753116 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:57:34.753097    5907 scope.go:122] "RemoveContainer" containerID="0600d617e4296fc89455c0ef1a13bb51be4788199da26b6cb1a17eaa7191495e"
Oct 30 19:57:34.761259 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:57:34.761241    5907 scope.go:122] "RemoveContainer" containerID="4e4b84711c327742a55bfb4cecccfeb517978e1d2a9892f2cabccad662cbeb04"
Oct 30 19:57:36.045633 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:57:36.045596    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-tg6wg"]
Oct 30 19:57:36.058232 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:57:36.058205    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-tg6wg"]
Oct 30 19:57:36.285987 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:57:36.285952    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0db10165-3412-4936-86b2-a7ec5e258915" path="/var/lib/kubelet/pods/0db10165-3412-4936-86b2-a7ec5e258915/volumes"
Oct 30 19:57:47.499936 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:57:47.499840    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:57:57.046365 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:57:57.046322    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-x7q49"]
Oct 30 19:57:57.059258 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:57:57.059225    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-x7q49"]
Oct 30 19:57:58.286656 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:57:58.286610    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e" path="/var/lib/kubelet/pods/5ba41ab1-4e12-4182-b1cc-aa5fc7e7af1e/volumes"
Oct 30 19:58:14.456658 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:14.456621    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:58:17.038887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:17.038169    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-d5fwl"]
Oct 30 19:58:17.060120 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:17.060082    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-d5fwl"]
Oct 30 19:58:17.545977 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:58:17.545924    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:58:18.288018 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:18.287982    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d28e37cc-059c-4e37-9d1f-5517215b3a73" path="/var/lib/kubelet/pods/d28e37cc-059c-4e37-9d1f-5517215b3a73/volumes"
Oct 30 19:58:18.525449 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:18.525405    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-ldvpq"]
Oct 30 19:58:18.525983 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:18.525963    5907 cpu_manager.go:414] "containerMap: removing container" podUID="e12a469d-c556-4761-a182-8c5498d7b180" containerName="revoke-keys"
Oct 30 19:58:18.526098 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:18.525984    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e12a469d-c556-4761-a182-8c5498d7b180" containerName="revoke-keys"
Oct 30 19:58:18.526161 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:18.526097    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="e12a469d-c556-4761-a182-8c5498d7b180" containerName="revoke-keys"
Oct 30 19:58:18.530607 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:18.530583    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-ldvpq"
Oct 30 19:58:18.550645 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:18.550558    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-ldvpq"]
Oct 30 19:58:18.560928 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:18.560877    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:58:18.560928 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:18.560922    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:58:18.669004 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:18.668965    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fh69\" (UniqueName: \"kubernetes.io/projected/b40bbeeb-a0e5-4909-8816-2b9e6054b8f7-kube-api-access-7fh69\") pod \"maas-api-revoke-sub-d576e68b-ldvpq\" (UID: \"b40bbeeb-a0e5-4909-8816-2b9e6054b8f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-ldvpq"
Oct 30 19:58:18.669185 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:18.669018    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b40bbeeb-a0e5-4909-8816-2b9e6054b8f7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d576e68b-ldvpq\" (UID: \"b40bbeeb-a0e5-4909-8816-2b9e6054b8f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-ldvpq"
Oct 30 19:58:18.669185 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:18.669143    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b40bbeeb-a0e5-4909-8816-2b9e6054b8f7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d576e68b-ldvpq\" (UID: \"b40bbeeb-a0e5-4909-8816-2b9e6054b8f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-ldvpq"
Oct 30 19:58:18.770756 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:18.770713    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fh69\" (UniqueName: \"kubernetes.io/projected/b40bbeeb-a0e5-4909-8816-2b9e6054b8f7-kube-api-access-7fh69\") pod \"maas-api-revoke-sub-d576e68b-ldvpq\" (UID: \"b40bbeeb-a0e5-4909-8816-2b9e6054b8f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-ldvpq"
Oct 30 19:58:18.770981 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:18.770773    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b40bbeeb-a0e5-4909-8816-2b9e6054b8f7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d576e68b-ldvpq\" (UID: \"b40bbeeb-a0e5-4909-8816-2b9e6054b8f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-ldvpq"
Oct 30 19:58:18.770981 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:18.770842    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b40bbeeb-a0e5-4909-8816-2b9e6054b8f7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d576e68b-ldvpq\" (UID: \"b40bbeeb-a0e5-4909-8816-2b9e6054b8f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-ldvpq"
Oct 30 19:58:18.771479 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:18.771449    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b40bbeeb-a0e5-4909-8816-2b9e6054b8f7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d576e68b-ldvpq\" (UID: \"b40bbeeb-a0e5-4909-8816-2b9e6054b8f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-ldvpq"
Oct 30 19:58:18.773392 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:18.773368    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b40bbeeb-a0e5-4909-8816-2b9e6054b8f7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d576e68b-ldvpq\" (UID: \"b40bbeeb-a0e5-4909-8816-2b9e6054b8f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-ldvpq"
Oct 30 19:58:18.791802 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:18.791770    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fh69\" (UniqueName: \"kubernetes.io/projected/b40bbeeb-a0e5-4909-8816-2b9e6054b8f7-kube-api-access-7fh69\") pod \"maas-api-revoke-sub-d576e68b-ldvpq\" (UID: \"b40bbeeb-a0e5-4909-8816-2b9e6054b8f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-ldvpq"
Oct 30 19:58:18.844367 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:18.844285    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-ldvpq"
Oct 30 19:58:19.146976 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:19.146948    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-ldvpq"]
Oct 30 19:58:19.150419 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:58:19.150385    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb40bbeeb_a0e5_4909_8816_2b9e6054b8f7.slice/crio-b1e9e69cd421f86592f012db0119202f57d9d109ea0ee93f9d9afadc0c9c2c10 WatchSource:0}: Error finding container b1e9e69cd421f86592f012db0119202f57d9d109ea0ee93f9d9afadc0c9c2c10: Status 404 returned error can't find the container with id b1e9e69cd421f86592f012db0119202f57d9d109ea0ee93f9d9afadc0c9c2c10
Oct 30 19:58:19.929887 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:19.929823    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-ldvpq" event={"ID":"b40bbeeb-a0e5-4909-8816-2b9e6054b8f7","Type":"ContainerStarted","Data":"4e641e38b9c03309d1de809168a710ad24f2f96e588987735a6d212ed916ad7e"}
Oct 30 19:58:19.930368 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:19.929910    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-ldvpq" event={"ID":"b40bbeeb-a0e5-4909-8816-2b9e6054b8f7","Type":"ContainerStarted","Data":"b1e9e69cd421f86592f012db0119202f57d9d109ea0ee93f9d9afadc0c9c2c10"}
Oct 30 19:58:19.973197 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:19.973140    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-ldvpq" podStartSLOduration=1.973126124 podStartE2EDuration="1.973126124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:58:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:58:19.970849521 +0000 UTC m=+2211.813781459" watchObservedRunningTime="2026-10-30 19:58:19.973126124 +0000 UTC m=+2211.816058060"
Oct 30 19:58:20.936775 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:20.936738    5907 generic.go:356] "Generic (PLEG): container finished" podID="b40bbeeb-a0e5-4909-8816-2b9e6054b8f7" containerID="4e641e38b9c03309d1de809168a710ad24f2f96e588987735a6d212ed916ad7e" exitCode=0
Oct 30 19:58:20.937299 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:20.936828    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-ldvpq" event={"ID":"b40bbeeb-a0e5-4909-8816-2b9e6054b8f7","Type":"ContainerDied","Data":"4e641e38b9c03309d1de809168a710ad24f2f96e588987735a6d212ed916ad7e"}
Oct 30 19:58:22.079313 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:22.079276    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-ldvpq"
Oct 30 19:58:22.208226 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:22.208142    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b40bbeeb-a0e5-4909-8816-2b9e6054b8f7-kube-api-access-7fh69\" (UniqueName: \"kubernetes.io/projected/b40bbeeb-a0e5-4909-8816-2b9e6054b8f7-kube-api-access-7fh69\") pod \"b40bbeeb-a0e5-4909-8816-2b9e6054b8f7\" (UID: \"b40bbeeb-a0e5-4909-8816-2b9e6054b8f7\") "
Oct 30 19:58:22.208226 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:22.208220    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b40bbeeb-a0e5-4909-8816-2b9e6054b8f7-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b40bbeeb-a0e5-4909-8816-2b9e6054b8f7-maas-api-xks-ca\") pod \"b40bbeeb-a0e5-4909-8816-2b9e6054b8f7\" (UID: \"b40bbeeb-a0e5-4909-8816-2b9e6054b8f7\") "
Oct 30 19:58:22.208447 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:22.208346    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b40bbeeb-a0e5-4909-8816-2b9e6054b8f7-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b40bbeeb-a0e5-4909-8816-2b9e6054b8f7-maas-api-openshift-ca\") pod \"b40bbeeb-a0e5-4909-8816-2b9e6054b8f7\" (UID: \"b40bbeeb-a0e5-4909-8816-2b9e6054b8f7\") "
Oct 30 19:58:22.208707 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:22.208685    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b40bbeeb-a0e5-4909-8816-2b9e6054b8f7-maas-api-openshift-ca" pod "b40bbeeb-a0e5-4909-8816-2b9e6054b8f7" (UID: "b40bbeeb-a0e5-4909-8816-2b9e6054b8f7"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:58:22.210374 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:22.210348    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b40bbeeb-a0e5-4909-8816-2b9e6054b8f7-kube-api-access-7fh69" pod "b40bbeeb-a0e5-4909-8816-2b9e6054b8f7" (UID: "b40bbeeb-a0e5-4909-8816-2b9e6054b8f7"). InnerVolumeSpecName "kube-api-access-7fh69". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:58:22.210480 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:22.210461    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b40bbeeb-a0e5-4909-8816-2b9e6054b8f7-maas-api-xks-ca" pod "b40bbeeb-a0e5-4909-8816-2b9e6054b8f7" (UID: "b40bbeeb-a0e5-4909-8816-2b9e6054b8f7"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:58:22.309705 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:22.309670    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fh69\" (UniqueName: \"kubernetes.io/projected/b40bbeeb-a0e5-4909-8816-2b9e6054b8f7-kube-api-access-7fh69\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:58:22.309705 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:22.309703    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b40bbeeb-a0e5-4909-8816-2b9e6054b8f7-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:58:22.309971 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:22.309713    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b40bbeeb-a0e5-4909-8816-2b9e6054b8f7-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:58:22.949083 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:22.949053    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-ldvpq"
Oct 30 19:58:22.949083 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:22.949071    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-ldvpq" event={"ID":"b40bbeeb-a0e5-4909-8816-2b9e6054b8f7","Type":"ContainerDied","Data":"b1e9e69cd421f86592f012db0119202f57d9d109ea0ee93f9d9afadc0c9c2c10"}
Oct 30 19:58:22.949647 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:22.949115    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b1e9e69cd421f86592f012db0119202f57d9d109ea0ee93f9d9afadc0c9c2c10"
Oct 30 19:58:29.854654 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:29.854614    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:58:34.798799 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:34.798756    5907 scope.go:122] "RemoveContainer" containerID="1c46fad432ed6a4989fca4ab03e3db0cf49ab26c9ffbedb5db1c11c5717f3fa9"
Oct 30 19:58:34.859134 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:34.859109    5907 scope.go:122] "RemoveContainer" containerID="2c07c8eb538c4c29cab8f8aa764320b532b90b18627d373fa820de1c68e4b589"
Oct 30 19:58:34.927440 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:34.926213    5907 scope.go:122] "RemoveContainer" containerID="c9729ee73cf998d8b9a15c7336d7dcd2162bee5e58c9eb05b0121f7da2c74e54"
Oct 30 19:58:35.035394 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:35.035363    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-kxwpx"]
Oct 30 19:58:35.066152 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:35.066122    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-kxwpx"]
Oct 30 19:58:35.342179 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:35.341990    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:58:36.286562 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:36.286528    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5df7d60f-e0d7-4c92-99c9-5593d805dd51" path="/var/lib/kubelet/pods/5df7d60f-e0d7-4c92-99c9-5593d805dd51/volumes"
Oct 30 19:58:39.921697 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:39.921660    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-6qj88"]
Oct 30 19:58:39.922138 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:39.922125    5907 cpu_manager.go:414] "containerMap: removing container" podUID="b40bbeeb-a0e5-4909-8816-2b9e6054b8f7" containerName="revoke-keys"
Oct 30 19:58:39.922187 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:39.922138    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b40bbeeb-a0e5-4909-8816-2b9e6054b8f7" containerName="revoke-keys"
Oct 30 19:58:39.922241 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:39.922232    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="b40bbeeb-a0e5-4909-8816-2b9e6054b8f7" containerName="revoke-keys"
Oct 30 19:58:39.927283 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:39.927265    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-6qj88"
Oct 30 19:58:39.945305 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:39.945280    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-6qj88"]
Oct 30 19:58:39.951908 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:39.951875    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x94jk\" (UniqueName: \"kubernetes.io/projected/2e11d300-dd1f-418b-bad6-bec630bfd7fc-kube-api-access-x94jk\") pod \"maas-api-revoke-sub-d2bc93bc-6qj88\" (UID: \"2e11d300-dd1f-418b-bad6-bec630bfd7fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-6qj88"
Oct 30 19:58:39.952041 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:39.952017    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2e11d300-dd1f-418b-bad6-bec630bfd7fc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d2bc93bc-6qj88\" (UID: \"2e11d300-dd1f-418b-bad6-bec630bfd7fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-6qj88"
Oct 30 19:58:39.952110 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:39.952089    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2e11d300-dd1f-418b-bad6-bec630bfd7fc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d2bc93bc-6qj88\" (UID: \"2e11d300-dd1f-418b-bad6-bec630bfd7fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-6qj88"
Oct 30 19:58:39.956624 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:39.956602    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:58:39.956727 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:39.956631    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:58:40.053257 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:40.053198    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x94jk\" (UniqueName: \"kubernetes.io/projected/2e11d300-dd1f-418b-bad6-bec630bfd7fc-kube-api-access-x94jk\") pod \"maas-api-revoke-sub-d2bc93bc-6qj88\" (UID: \"2e11d300-dd1f-418b-bad6-bec630bfd7fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-6qj88"
Oct 30 19:58:40.053461 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:40.053316    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2e11d300-dd1f-418b-bad6-bec630bfd7fc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d2bc93bc-6qj88\" (UID: \"2e11d300-dd1f-418b-bad6-bec630bfd7fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-6qj88"
Oct 30 19:58:40.053461 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:40.053348    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2e11d300-dd1f-418b-bad6-bec630bfd7fc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d2bc93bc-6qj88\" (UID: \"2e11d300-dd1f-418b-bad6-bec630bfd7fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-6qj88"
Oct 30 19:58:40.054062 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:40.054038    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2e11d300-dd1f-418b-bad6-bec630bfd7fc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d2bc93bc-6qj88\" (UID: \"2e11d300-dd1f-418b-bad6-bec630bfd7fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-6qj88"
Oct 30 19:58:40.055820 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:40.055797    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2e11d300-dd1f-418b-bad6-bec630bfd7fc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d2bc93bc-6qj88\" (UID: \"2e11d300-dd1f-418b-bad6-bec630bfd7fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-6qj88"
Oct 30 19:58:40.073581 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:40.073552    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x94jk\" (UniqueName: \"kubernetes.io/projected/2e11d300-dd1f-418b-bad6-bec630bfd7fc-kube-api-access-x94jk\") pod \"maas-api-revoke-sub-d2bc93bc-6qj88\" (UID: \"2e11d300-dd1f-418b-bad6-bec630bfd7fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-6qj88"
Oct 30 19:58:40.241337 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:40.241256    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-6qj88"
Oct 30 19:58:40.541014 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:40.540945    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-6qj88"]
Oct 30 19:58:40.543405 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:58:40.543377    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e11d300_dd1f_418b_bad6_bec630bfd7fc.slice/crio-7dd10549e0d46c0fdf2f3b27418dd2aa0d04b46a0adc091b17ac9dab5f4ba6ce WatchSource:0}: Error finding container 7dd10549e0d46c0fdf2f3b27418dd2aa0d04b46a0adc091b17ac9dab5f4ba6ce: Status 404 returned error can't find the container with id 7dd10549e0d46c0fdf2f3b27418dd2aa0d04b46a0adc091b17ac9dab5f4ba6ce
Oct 30 19:58:41.134390 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:41.134357    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-6qj88" event={"ID":"2e11d300-dd1f-418b-bad6-bec630bfd7fc","Type":"ContainerStarted","Data":"f5b5b9f5756f42ca5d0e58c4ae53fdd9aa236baeecdf4269cd062764c1daac4b"}
Oct 30 19:58:41.134843 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:41.134415    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-6qj88" event={"ID":"2e11d300-dd1f-418b-bad6-bec630bfd7fc","Type":"ContainerStarted","Data":"7dd10549e0d46c0fdf2f3b27418dd2aa0d04b46a0adc091b17ac9dab5f4ba6ce"}
Oct 30 19:58:41.176126 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:41.176080    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-6qj88" podStartSLOduration=2.176066125 podStartE2EDuration="2.176066125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:58:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:58:41.174471507 +0000 UTC m=+2233.017403459" watchObservedRunningTime="2026-10-30 19:58:41.176066125 +0000 UTC m=+2233.018998064"
Oct 30 19:58:42.141306 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:42.141268    5907 generic.go:356] "Generic (PLEG): container finished" podID="2e11d300-dd1f-418b-bad6-bec630bfd7fc" containerID="f5b5b9f5756f42ca5d0e58c4ae53fdd9aa236baeecdf4269cd062764c1daac4b" exitCode=0
Oct 30 19:58:42.141769 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:42.141362    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-6qj88" event={"ID":"2e11d300-dd1f-418b-bad6-bec630bfd7fc","Type":"ContainerDied","Data":"f5b5b9f5756f42ca5d0e58c4ae53fdd9aa236baeecdf4269cd062764c1daac4b"}
Oct 30 19:58:43.280491 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:43.280466    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-6qj88"
Oct 30 19:58:43.388411 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:43.388378    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2e11d300-dd1f-418b-bad6-bec630bfd7fc-kube-api-access-x94jk\" (UniqueName: \"kubernetes.io/projected/2e11d300-dd1f-418b-bad6-bec630bfd7fc-kube-api-access-x94jk\") pod \"2e11d300-dd1f-418b-bad6-bec630bfd7fc\" (UID: \"2e11d300-dd1f-418b-bad6-bec630bfd7fc\") "
Oct 30 19:58:43.388612 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:43.388478    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2e11d300-dd1f-418b-bad6-bec630bfd7fc-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2e11d300-dd1f-418b-bad6-bec630bfd7fc-maas-api-xks-ca\") pod \"2e11d300-dd1f-418b-bad6-bec630bfd7fc\" (UID: \"2e11d300-dd1f-418b-bad6-bec630bfd7fc\") "
Oct 30 19:58:43.388612 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:43.388554    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2e11d300-dd1f-418b-bad6-bec630bfd7fc-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2e11d300-dd1f-418b-bad6-bec630bfd7fc-maas-api-openshift-ca\") pod \"2e11d300-dd1f-418b-bad6-bec630bfd7fc\" (UID: \"2e11d300-dd1f-418b-bad6-bec630bfd7fc\") "
Oct 30 19:58:43.389167 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:43.389134    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e11d300-dd1f-418b-bad6-bec630bfd7fc-maas-api-openshift-ca" pod "2e11d300-dd1f-418b-bad6-bec630bfd7fc" (UID: "2e11d300-dd1f-418b-bad6-bec630bfd7fc"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:58:43.390633 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:43.390041    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2e11d300-dd1f-418b-bad6-bec630bfd7fc-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:58:43.391266 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:43.391242    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e11d300-dd1f-418b-bad6-bec630bfd7fc-kube-api-access-x94jk" pod "2e11d300-dd1f-418b-bad6-bec630bfd7fc" (UID: "2e11d300-dd1f-418b-bad6-bec630bfd7fc"). InnerVolumeSpecName "kube-api-access-x94jk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:58:43.391384 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:43.391277    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e11d300-dd1f-418b-bad6-bec630bfd7fc-maas-api-xks-ca" pod "2e11d300-dd1f-418b-bad6-bec630bfd7fc" (UID: "2e11d300-dd1f-418b-bad6-bec630bfd7fc"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:58:43.491497 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:43.491465    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x94jk\" (UniqueName: \"kubernetes.io/projected/2e11d300-dd1f-418b-bad6-bec630bfd7fc-kube-api-access-x94jk\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:58:43.491497 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:43.491495    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2e11d300-dd1f-418b-bad6-bec630bfd7fc-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:58:44.154374 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:44.154342    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-6qj88"
Oct 30 19:58:44.154571 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:44.154342    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-6qj88" event={"ID":"2e11d300-dd1f-418b-bad6-bec630bfd7fc","Type":"ContainerDied","Data":"7dd10549e0d46c0fdf2f3b27418dd2aa0d04b46a0adc091b17ac9dab5f4ba6ce"}
Oct 30 19:58:44.154571 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:44.154465    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7dd10549e0d46c0fdf2f3b27418dd2aa0d04b46a0adc091b17ac9dab5f4ba6ce"
Oct 30 19:58:47.501190 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:58:47.501145    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:58:51.187509 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:58:51.187471    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:59:00.765970 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:00.765933    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:59:15.489723 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:15.489631    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:59:17.500920 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:59:17.500848    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:59:31.488025 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:31.487994    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:59:34.032758 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:34.032721    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kcqxp"]
Oct 30 19:59:34.045925 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:34.045893    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-kcqxp"]
Oct 30 19:59:34.286264 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:34.286186    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="919c056f-1a01-4b8f-a6ee-0f49f79e64ad" path="/var/lib/kubelet/pods/919c056f-1a01-4b8f-a6ee-0f49f79e64ad/volumes"
Oct 30 19:59:35.155990 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:35.155957    5907 scope.go:122] "RemoveContainer" containerID="59364f99a98c671279f88700ba4fd2bafdfcb70f758050e0d72153afe2ca9b0a"
Oct 30 19:59:35.164976 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:35.164960    5907 scope.go:122] "RemoveContainer" containerID="503f3dfcc101693a55a839ca553a59ff7817d21d317935d920caf1608e5448c4"
Oct 30 19:59:35.172999 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:35.172978    5907 scope.go:122] "RemoveContainer" containerID="2ccb89b5be17451678f24c1daf17f660cc39a3dc0938e1b23dc13a560e81d06d"
Oct 30 19:59:43.611118 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:43.611087    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f62d239e-rf2dm"]
Oct 30 19:59:43.611530 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:43.611515    5907 cpu_manager.go:414] "containerMap: removing container" podUID="2e11d300-dd1f-418b-bad6-bec630bfd7fc" containerName="revoke-keys"
Oct 30 19:59:43.611577 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:43.611531    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2e11d300-dd1f-418b-bad6-bec630bfd7fc" containerName="revoke-keys"
Oct 30 19:59:43.611643 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:43.611633    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="2e11d300-dd1f-418b-bad6-bec630bfd7fc" containerName="revoke-keys"
Oct 30 19:59:43.616623 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:43.616606    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f62d239e-rf2dm"
Oct 30 19:59:43.647232 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:43.647205    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:59:43.647232 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:43.647214    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:59:43.649174 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:43.649156    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f62d239e-rf2dm"]
Oct 30 19:59:43.730373 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:43.730341    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/00286d05-f903-48ca-ad1a-1415e4629850-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f62d239e-rf2dm\" (UID: \"00286d05-f903-48ca-ad1a-1415e4629850\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f62d239e-rf2dm"
Oct 30 19:59:43.730531 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:43.730386    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/00286d05-f903-48ca-ad1a-1415e4629850-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f62d239e-rf2dm\" (UID: \"00286d05-f903-48ca-ad1a-1415e4629850\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f62d239e-rf2dm"
Oct 30 19:59:43.730580 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:43.730559    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z85cn\" (UniqueName: \"kubernetes.io/projected/00286d05-f903-48ca-ad1a-1415e4629850-kube-api-access-z85cn\") pod \"maas-api-revoke-sub-f62d239e-rf2dm\" (UID: \"00286d05-f903-48ca-ad1a-1415e4629850\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f62d239e-rf2dm"
Oct 30 19:59:43.831839 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:43.831810    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/00286d05-f903-48ca-ad1a-1415e4629850-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f62d239e-rf2dm\" (UID: \"00286d05-f903-48ca-ad1a-1415e4629850\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f62d239e-rf2dm"
Oct 30 19:59:43.831839 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:43.831841    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/00286d05-f903-48ca-ad1a-1415e4629850-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f62d239e-rf2dm\" (UID: \"00286d05-f903-48ca-ad1a-1415e4629850\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f62d239e-rf2dm"
Oct 30 19:59:43.832048 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:43.831958    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z85cn\" (UniqueName: \"kubernetes.io/projected/00286d05-f903-48ca-ad1a-1415e4629850-kube-api-access-z85cn\") pod \"maas-api-revoke-sub-f62d239e-rf2dm\" (UID: \"00286d05-f903-48ca-ad1a-1415e4629850\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f62d239e-rf2dm"
Oct 30 19:59:43.832580 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:43.832560    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/00286d05-f903-48ca-ad1a-1415e4629850-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f62d239e-rf2dm\" (UID: \"00286d05-f903-48ca-ad1a-1415e4629850\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f62d239e-rf2dm"
Oct 30 19:59:43.834264 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:43.834245    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/00286d05-f903-48ca-ad1a-1415e4629850-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f62d239e-rf2dm\" (UID: \"00286d05-f903-48ca-ad1a-1415e4629850\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f62d239e-rf2dm"
Oct 30 19:59:43.853621 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:43.853594    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z85cn\" (UniqueName: \"kubernetes.io/projected/00286d05-f903-48ca-ad1a-1415e4629850-kube-api-access-z85cn\") pod \"maas-api-revoke-sub-f62d239e-rf2dm\" (UID: \"00286d05-f903-48ca-ad1a-1415e4629850\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f62d239e-rf2dm"
Oct 30 19:59:43.929282 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:43.929218    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f62d239e-rf2dm"
Oct 30 19:59:44.231514 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:44.231440    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f62d239e-rf2dm"]
Oct 30 19:59:44.234350 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:59:44.234321    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00286d05_f903_48ca_ad1a_1415e4629850.slice/crio-435ccf8b52ff4f35260abc84e52eaefc01e5f5ac1b9f9ac77b3d26308cbee5a7 WatchSource:0}: Error finding container 435ccf8b52ff4f35260abc84e52eaefc01e5f5ac1b9f9ac77b3d26308cbee5a7: Status 404 returned error can't find the container with id 435ccf8b52ff4f35260abc84e52eaefc01e5f5ac1b9f9ac77b3d26308cbee5a7
Oct 30 19:59:44.505556 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:44.505469    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f62d239e-rf2dm" event={"ID":"00286d05-f903-48ca-ad1a-1415e4629850","Type":"ContainerStarted","Data":"435ccf8b52ff4f35260abc84e52eaefc01e5f5ac1b9f9ac77b3d26308cbee5a7"}
Oct 30 19:59:45.031605 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:45.031574    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-zpswv"]
Oct 30 19:59:45.044659 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:45.044630    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-zpswv"]
Oct 30 19:59:45.511649 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:45.511611    5907 generic.go:356] "Generic (PLEG): container finished" podID="00286d05-f903-48ca-ad1a-1415e4629850" containerID="c5389877a0b19e6b08dbec0206f4923eec46c7b4d4671ac83f411a661b1d29fd" exitCode=0
Oct 30 19:59:45.511892 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:45.511666    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f62d239e-rf2dm" event={"ID":"00286d05-f903-48ca-ad1a-1415e4629850","Type":"ContainerDied","Data":"c5389877a0b19e6b08dbec0206f4923eec46c7b4d4671ac83f411a661b1d29fd"}
Oct 30 19:59:46.291498 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:46.291453    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d27dd847-6b5a-4645-a571-c5b7271fdf88" path="/var/lib/kubelet/pods/d27dd847-6b5a-4645-a571-c5b7271fdf88/volumes"
Oct 30 19:59:46.653293 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:46.653266    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f62d239e-rf2dm"
Oct 30 19:59:46.763360 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:46.763327    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/00286d05-f903-48ca-ad1a-1415e4629850-kube-api-access-z85cn\" (UniqueName: \"kubernetes.io/projected/00286d05-f903-48ca-ad1a-1415e4629850-kube-api-access-z85cn\") pod \"00286d05-f903-48ca-ad1a-1415e4629850\" (UID: \"00286d05-f903-48ca-ad1a-1415e4629850\") "
Oct 30 19:59:46.763360 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:46.763363    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/00286d05-f903-48ca-ad1a-1415e4629850-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/00286d05-f903-48ca-ad1a-1415e4629850-maas-api-openshift-ca\") pod \"00286d05-f903-48ca-ad1a-1415e4629850\" (UID: \"00286d05-f903-48ca-ad1a-1415e4629850\") "
Oct 30 19:59:46.763630 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:46.763431    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/00286d05-f903-48ca-ad1a-1415e4629850-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/00286d05-f903-48ca-ad1a-1415e4629850-maas-api-xks-ca\") pod \"00286d05-f903-48ca-ad1a-1415e4629850\" (UID: \"00286d05-f903-48ca-ad1a-1415e4629850\") "
Oct 30 19:59:46.763832 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:46.763807    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00286d05-f903-48ca-ad1a-1415e4629850-maas-api-openshift-ca" pod "00286d05-f903-48ca-ad1a-1415e4629850" (UID: "00286d05-f903-48ca-ad1a-1415e4629850"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:59:46.763971 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:46.763951    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/00286d05-f903-48ca-ad1a-1415e4629850-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:59:46.765664 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:46.765644    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00286d05-f903-48ca-ad1a-1415e4629850-maas-api-xks-ca" pod "00286d05-f903-48ca-ad1a-1415e4629850" (UID: "00286d05-f903-48ca-ad1a-1415e4629850"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:59:46.765715 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:46.765682    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00286d05-f903-48ca-ad1a-1415e4629850-kube-api-access-z85cn" pod "00286d05-f903-48ca-ad1a-1415e4629850" (UID: "00286d05-f903-48ca-ad1a-1415e4629850"). InnerVolumeSpecName "kube-api-access-z85cn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:59:46.864635 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:46.864602    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z85cn\" (UniqueName: \"kubernetes.io/projected/00286d05-f903-48ca-ad1a-1415e4629850-kube-api-access-z85cn\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:59:46.864635 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:46.864632    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/00286d05-f903-48ca-ad1a-1415e4629850-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:59:47.500270 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:59:47.500234    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 19:59:47.522762 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:47.522729    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f62d239e-rf2dm"
Oct 30 19:59:47.522941 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:47.522760    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f62d239e-rf2dm" event={"ID":"00286d05-f903-48ca-ad1a-1415e4629850","Type":"ContainerDied","Data":"435ccf8b52ff4f35260abc84e52eaefc01e5f5ac1b9f9ac77b3d26308cbee5a7"}
Oct 30 19:59:47.522941 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:47.522808    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="435ccf8b52ff4f35260abc84e52eaefc01e5f5ac1b9f9ac77b3d26308cbee5a7"
Oct 30 19:59:54.671953 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:54.671914    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-a25f47-sfr9t"]
Oct 30 19:59:54.672440 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:54.672398    5907 cpu_manager.go:414] "containerMap: removing container" podUID="00286d05-f903-48ca-ad1a-1415e4629850" containerName="revoke-keys"
Oct 30 19:59:54.672440 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:54.672410    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="00286d05-f903-48ca-ad1a-1415e4629850" containerName="revoke-keys"
Oct 30 19:59:54.672526 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:54.672488    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="00286d05-f903-48ca-ad1a-1415e4629850" containerName="revoke-keys"
Oct 30 19:59:54.675665 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:54.675646    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-a25f47-sfr9t"
Oct 30 19:59:54.696746 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:54.696716    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-a25f47-sfr9t"]
Oct 30 19:59:54.707350 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:54.707318    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:59:54.707473 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:54.707398    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 19:59:54.745320 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:54.745294    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgzmx\" (UniqueName: \"kubernetes.io/projected/99dab13b-4421-4d78-9715-4a8a04926a06-kube-api-access-vgzmx\") pod \"maas-api-revoke-keys-e2e-worker-single-main-a25f47-sfr9t\" (UID: \"99dab13b-4421-4d78-9715-4a8a04926a06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-a25f47-sfr9t"
Oct 30 19:59:54.745458 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:54.745349    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/99dab13b-4421-4d78-9715-4a8a04926a06-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-a25f47-sfr9t\" (UID: \"99dab13b-4421-4d78-9715-4a8a04926a06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-a25f47-sfr9t"
Oct 30 19:59:54.745503 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:54.745454    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/99dab13b-4421-4d78-9715-4a8a04926a06-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-a25f47-sfr9t\" (UID: \"99dab13b-4421-4d78-9715-4a8a04926a06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-a25f47-sfr9t"
Oct 30 19:59:54.846624 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:54.846578    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgzmx\" (UniqueName: \"kubernetes.io/projected/99dab13b-4421-4d78-9715-4a8a04926a06-kube-api-access-vgzmx\") pod \"maas-api-revoke-keys-e2e-worker-single-main-a25f47-sfr9t\" (UID: \"99dab13b-4421-4d78-9715-4a8a04926a06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-a25f47-sfr9t"
Oct 30 19:59:54.846813 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:54.846644    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/99dab13b-4421-4d78-9715-4a8a04926a06-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-a25f47-sfr9t\" (UID: \"99dab13b-4421-4d78-9715-4a8a04926a06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-a25f47-sfr9t"
Oct 30 19:59:54.846813 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:54.846689    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/99dab13b-4421-4d78-9715-4a8a04926a06-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-a25f47-sfr9t\" (UID: \"99dab13b-4421-4d78-9715-4a8a04926a06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-a25f47-sfr9t"
Oct 30 19:59:54.847426 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:54.847357    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/99dab13b-4421-4d78-9715-4a8a04926a06-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-a25f47-sfr9t\" (UID: \"99dab13b-4421-4d78-9715-4a8a04926a06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-a25f47-sfr9t"
Oct 30 19:59:54.849135 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:54.849111    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/99dab13b-4421-4d78-9715-4a8a04926a06-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-a25f47-sfr9t\" (UID: \"99dab13b-4421-4d78-9715-4a8a04926a06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-a25f47-sfr9t"
Oct 30 19:59:54.867573 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:54.867548    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgzmx\" (UniqueName: \"kubernetes.io/projected/99dab13b-4421-4d78-9715-4a8a04926a06-kube-api-access-vgzmx\") pod \"maas-api-revoke-keys-e2e-worker-single-main-a25f47-sfr9t\" (UID: \"99dab13b-4421-4d78-9715-4a8a04926a06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-a25f47-sfr9t"
Oct 30 19:59:54.988060 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:54.987978    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-a25f47-sfr9t"
Oct 30 19:59:55.191915 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:55.191881    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 19:59:55.288661 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:55.288629    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-a25f47-sfr9t"]
Oct 30 19:59:55.291373 ip-10-0-132-152 kubenswrapper[5907]: W1030 19:59:55.291339    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99dab13b_4421_4d78_9715_4a8a04926a06.slice/crio-5553695f25e0abaecbe27b2a6e389822cc255b319e022f723831b5286232e5ba WatchSource:0}: Error finding container 5553695f25e0abaecbe27b2a6e389822cc255b319e022f723831b5286232e5ba: Status 404 returned error can't find the container with id 5553695f25e0abaecbe27b2a6e389822cc255b319e022f723831b5286232e5ba
Oct 30 19:59:55.563500 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:55.563410    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-a25f47-sfr9t" event={"ID":"99dab13b-4421-4d78-9715-4a8a04926a06","Type":"ContainerStarted","Data":"5553695f25e0abaecbe27b2a6e389822cc255b319e022f723831b5286232e5ba"}
Oct 30 19:59:56.570676 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:56.570586    5907 generic.go:356] "Generic (PLEG): container finished" podID="99dab13b-4421-4d78-9715-4a8a04926a06" containerID="ae6c0644544bcb0275f8a7444678eb8e5315b3103e34aa68db7a3ad108fb274d" exitCode=0
Oct 30 19:59:56.570676 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:56.570642    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-a25f47-sfr9t" event={"ID":"99dab13b-4421-4d78-9715-4a8a04926a06","Type":"ContainerDied","Data":"ae6c0644544bcb0275f8a7444678eb8e5315b3103e34aa68db7a3ad108fb274d"}
Oct 30 19:59:57.716097 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:57.716068    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-a25f47-sfr9t"
Oct 30 19:59:57.777680 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:57.777644    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/99dab13b-4421-4d78-9715-4a8a04926a06-kube-api-access-vgzmx\" (UniqueName: \"kubernetes.io/projected/99dab13b-4421-4d78-9715-4a8a04926a06-kube-api-access-vgzmx\") pod \"99dab13b-4421-4d78-9715-4a8a04926a06\" (UID: \"99dab13b-4421-4d78-9715-4a8a04926a06\") "
Oct 30 19:59:57.777827 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:57.777712    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/99dab13b-4421-4d78-9715-4a8a04926a06-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/99dab13b-4421-4d78-9715-4a8a04926a06-maas-api-openshift-ca\") pod \"99dab13b-4421-4d78-9715-4a8a04926a06\" (UID: \"99dab13b-4421-4d78-9715-4a8a04926a06\") "
Oct 30 19:59:57.777909 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:57.777849    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/99dab13b-4421-4d78-9715-4a8a04926a06-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/99dab13b-4421-4d78-9715-4a8a04926a06-maas-api-xks-ca\") pod \"99dab13b-4421-4d78-9715-4a8a04926a06\" (UID: \"99dab13b-4421-4d78-9715-4a8a04926a06\") "
Oct 30 19:59:57.778159 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:57.778131    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99dab13b-4421-4d78-9715-4a8a04926a06-maas-api-openshift-ca" pod "99dab13b-4421-4d78-9715-4a8a04926a06" (UID: "99dab13b-4421-4d78-9715-4a8a04926a06"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:59:57.778281 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:57.778252    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/99dab13b-4421-4d78-9715-4a8a04926a06-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:59:57.779907 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:57.779849    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99dab13b-4421-4d78-9715-4a8a04926a06-kube-api-access-vgzmx" pod "99dab13b-4421-4d78-9715-4a8a04926a06" (UID: "99dab13b-4421-4d78-9715-4a8a04926a06"). InnerVolumeSpecName "kube-api-access-vgzmx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:59:57.780110 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:57.780088    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99dab13b-4421-4d78-9715-4a8a04926a06-maas-api-xks-ca" pod "99dab13b-4421-4d78-9715-4a8a04926a06" (UID: "99dab13b-4421-4d78-9715-4a8a04926a06"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:59:57.879348 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:57.879323    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/99dab13b-4421-4d78-9715-4a8a04926a06-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:59:57.879348 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:57.879351    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vgzmx\" (UniqueName: \"kubernetes.io/projected/99dab13b-4421-4d78-9715-4a8a04926a06-kube-api-access-vgzmx\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 19:59:58.317578 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:59:58.317548    5907 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99dab13b_4421_4d78_9715_4a8a04926a06.slice\": RecentStats: unable to find data in memory cache]"
Oct 30 19:59:58.317753 ip-10-0-132-152 kubenswrapper[5907]: E1030 19:59:58.317650    5907 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99dab13b_4421_4d78_9715_4a8a04926a06.slice\": RecentStats: unable to find data in memory cache]"
Oct 30 19:59:58.583283 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:58.583196    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-a25f47-sfr9t"
Oct 30 19:59:58.583283 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:58.583234    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-a25f47-sfr9t" event={"ID":"99dab13b-4421-4d78-9715-4a8a04926a06","Type":"ContainerDied","Data":"5553695f25e0abaecbe27b2a6e389822cc255b319e022f723831b5286232e5ba"}
Oct 30 19:59:58.583478 ip-10-0-132-152 kubenswrapper[5907]: I1030 19:59:58.583298    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5553695f25e0abaecbe27b2a6e389822cc255b319e022f723831b5286232e5ba"
Oct 30 20:00:00.140779 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:00.140744    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-a25f47-29889846gk69"]
Oct 30 20:00:00.141283 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:00.141264    5907 cpu_manager.go:414] "containerMap: removing container" podUID="99dab13b-4421-4d78-9715-4a8a04926a06" containerName="revoke-keys"
Oct 30 20:00:00.141334 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:00.141286    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="99dab13b-4421-4d78-9715-4a8a04926a06" containerName="revoke-keys"
Oct 30 20:00:00.141390 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:00.141381    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="99dab13b-4421-4d78-9715-4a8a04926a06" containerName="revoke-keys"
Oct 30 20:00:00.144893 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:00.144849    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-a25f47-29889846gk69"
Oct 30 20:00:00.164256 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:00.164239    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 20:00:00.180301 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:00.180268    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-a25f47-29889846gk69"]
Oct 30 20:00:00.201673 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:00.201648    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/534df7f5-7ced-42ea-b56b-a573ee8ca0c3-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-single-main-a25f47-29889846gk69\" (UID: \"534df7f5-7ced-42ea-b56b-a573ee8ca0c3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-a25f47-29889846gk69"
Oct 30 20:00:00.201811 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:00.201679    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jq7np\" (UniqueName: \"kubernetes.io/projected/534df7f5-7ced-42ea-b56b-a573ee8ca0c3-kube-api-access-jq7np\") pod \"maas-api-key-cleanup-e2e-worker-single-main-a25f47-29889846gk69\" (UID: \"534df7f5-7ced-42ea-b56b-a573ee8ca0c3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-a25f47-29889846gk69"
Oct 30 20:00:00.302825 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:00.302792    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/534df7f5-7ced-42ea-b56b-a573ee8ca0c3-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-single-main-a25f47-29889846gk69\" (UID: \"534df7f5-7ced-42ea-b56b-a573ee8ca0c3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-a25f47-29889846gk69"
Oct 30 20:00:00.302825 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:00.302827    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jq7np\" (UniqueName: \"kubernetes.io/projected/534df7f5-7ced-42ea-b56b-a573ee8ca0c3-kube-api-access-jq7np\") pod \"maas-api-key-cleanup-e2e-worker-single-main-a25f47-29889846gk69\" (UID: \"534df7f5-7ced-42ea-b56b-a573ee8ca0c3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-a25f47-29889846gk69"
Oct 30 20:00:00.303500 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:00.303480    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/534df7f5-7ced-42ea-b56b-a573ee8ca0c3-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-single-main-a25f47-29889846gk69\" (UID: \"534df7f5-7ced-42ea-b56b-a573ee8ca0c3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-a25f47-29889846gk69"
Oct 30 20:00:00.340265 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:00.339043    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jq7np\" (UniqueName: \"kubernetes.io/projected/534df7f5-7ced-42ea-b56b-a573ee8ca0c3-kube-api-access-jq7np\") pod \"maas-api-key-cleanup-e2e-worker-single-main-a25f47-29889846gk69\" (UID: \"534df7f5-7ced-42ea-b56b-a573ee8ca0c3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-a25f47-29889846gk69"
Oct 30 20:00:00.458515 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:00.458441    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-a25f47-29889846gk69"
Oct 30 20:00:00.764761 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:00.764683    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-a25f47-29889846gk69"]
Oct 30 20:00:01.601222 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:01.601190    5907 generic.go:356] "Generic (PLEG): container finished" podID="534df7f5-7ced-42ea-b56b-a573ee8ca0c3" containerID="a21afe1e751e6796c9c9619b2bc48e5de519973217602f10ce92a5ad2fe6efb6" exitCode=60
Oct 30 20:00:01.601632 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:01.601283    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-a25f47-29889846gk69" event={"ID":"534df7f5-7ced-42ea-b56b-a573ee8ca0c3","Type":"ContainerDied","Data":"a21afe1e751e6796c9c9619b2bc48e5de519973217602f10ce92a5ad2fe6efb6"}
Oct 30 20:00:01.601632 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:01.601344    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-a25f47-29889846gk69" event={"ID":"534df7f5-7ced-42ea-b56b-a573ee8ca0c3","Type":"ContainerStarted","Data":"cd92c72840e4ceb6471f0e2903f8f77071da6fbdfed35797f84ae409b1c05fa4"}
Oct 30 20:00:01.601632 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:01.601537    5907 scope.go:122] "RemoveContainer" containerID="a21afe1e751e6796c9c9619b2bc48e5de519973217602f10ce92a5ad2fe6efb6"
Oct 30 20:00:02.609601 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:02.609546    5907 generic.go:356] "Generic (PLEG): container finished" podID="534df7f5-7ced-42ea-b56b-a573ee8ca0c3" containerID="615a624355e8109e207e1bfc16f052feef8e44ff295c354d7746f31dca5f5428" exitCode=60
Oct 30 20:00:02.610085 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:02.609604    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-a25f47-29889846gk69" event={"ID":"534df7f5-7ced-42ea-b56b-a573ee8ca0c3","Type":"ContainerDied","Data":"615a624355e8109e207e1bfc16f052feef8e44ff295c354d7746f31dca5f5428"}
Oct 30 20:00:02.610085 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:02.609685    5907 scope.go:122] "RemoveContainer" containerID="a21afe1e751e6796c9c9619b2bc48e5de519973217602f10ce92a5ad2fe6efb6"
Oct 30 20:00:02.610085 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:02.609978    5907 scope.go:122] "RemoveContainer" containerID="615a624355e8109e207e1bfc16f052feef8e44ff295c354d7746f31dca5f5428"
Oct 30 20:00:02.610287 ip-10-0-132-152 kubenswrapper[5907]: E1030 20:00:02.610267    5907 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-single-main-a25f47-29889846gk69_odh-ai-gateway-infra(534df7f5-7ced-42ea-b56b-a573ee8ca0c3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-a25f47-29889846gk69" podUID="534df7f5-7ced-42ea-b56b-a573ee8ca0c3"
Oct 30 20:00:03.616991 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:03.616965    5907 scope.go:122] "RemoveContainer" containerID="615a624355e8109e207e1bfc16f052feef8e44ff295c354d7746f31dca5f5428"
Oct 30 20:00:03.617390 ip-10-0-132-152 kubenswrapper[5907]: E1030 20:00:03.617224    5907 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-single-main-a25f47-29889846gk69_odh-ai-gateway-infra(534df7f5-7ced-42ea-b56b-a573ee8ca0c3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-a25f47-29889846gk69" podUID="534df7f5-7ced-42ea-b56b-a573ee8ca0c3"
Oct 30 20:00:06.403838 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:06.403804    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-a25f47-29889846gk69"]
Oct 30 20:00:06.544420 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:06.544391    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-a25f47-29889846gk69"
Oct 30 20:00:06.633264 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:06.633236    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-a25f47-29889846gk69"
Oct 30 20:00:06.633443 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:06.633270    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-a25f47-29889846gk69" event={"ID":"534df7f5-7ced-42ea-b56b-a573ee8ca0c3","Type":"ContainerDied","Data":"cd92c72840e4ceb6471f0e2903f8f77071da6fbdfed35797f84ae409b1c05fa4"}
Oct 30 20:00:06.633443 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:06.633324    5907 scope.go:122] "RemoveContainer" containerID="615a624355e8109e207e1bfc16f052feef8e44ff295c354d7746f31dca5f5428"
Oct 30 20:00:06.680852 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:06.680767    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/534df7f5-7ced-42ea-b56b-a573ee8ca0c3-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/534df7f5-7ced-42ea-b56b-a573ee8ca0c3-maas-api-service-ca\") pod \"534df7f5-7ced-42ea-b56b-a573ee8ca0c3\" (UID: \"534df7f5-7ced-42ea-b56b-a573ee8ca0c3\") "
Oct 30 20:00:06.680852 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:06.680829    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/534df7f5-7ced-42ea-b56b-a573ee8ca0c3-kube-api-access-jq7np\" (UniqueName: \"kubernetes.io/projected/534df7f5-7ced-42ea-b56b-a573ee8ca0c3-kube-api-access-jq7np\") pod \"534df7f5-7ced-42ea-b56b-a573ee8ca0c3\" (UID: \"534df7f5-7ced-42ea-b56b-a573ee8ca0c3\") "
Oct 30 20:00:06.681221 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:06.681195    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/534df7f5-7ced-42ea-b56b-a573ee8ca0c3-maas-api-service-ca" pod "534df7f5-7ced-42ea-b56b-a573ee8ca0c3" (UID: "534df7f5-7ced-42ea-b56b-a573ee8ca0c3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:00:06.704054 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:06.704016    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/534df7f5-7ced-42ea-b56b-a573ee8ca0c3-kube-api-access-jq7np" pod "534df7f5-7ced-42ea-b56b-a573ee8ca0c3" (UID: "534df7f5-7ced-42ea-b56b-a573ee8ca0c3"). InnerVolumeSpecName "kube-api-access-jq7np". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:00:06.782005 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:06.781966    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/534df7f5-7ced-42ea-b56b-a573ee8ca0c3-maas-api-service-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 20:00:06.782005 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:06.781996    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jq7np\" (UniqueName: \"kubernetes.io/projected/534df7f5-7ced-42ea-b56b-a573ee8ca0c3-kube-api-access-jq7np\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 20:00:06.997733 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:06.997695    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-a25f47-29889846gk69"]
Oct 30 20:00:07.015636 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:07.015602    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-single-main-a25f47-29889846gk69"]
Oct 30 20:00:07.082298 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:07.082264    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-a25f47-sfr9t"]
Oct 30 20:00:07.086398 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:07.086371    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-a25f47-sfr9t"]
Oct 30 20:00:08.286780 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:08.286741    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="534df7f5-7ced-42ea-b56b-a573ee8ca0c3" path="/var/lib/kubelet/pods/534df7f5-7ced-42ea-b56b-a573ee8ca0c3/volumes"
Oct 30 20:00:08.287387 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:08.287365    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="99dab13b-4421-4d78-9715-4a8a04926a06" path="/var/lib/kubelet/pods/99dab13b-4421-4d78-9715-4a8a04926a06/volumes"
Oct 30 20:00:08.886887 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:08.884514    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 20:00:17.500938 ip-10-0-132-152 kubenswrapper[5907]: E1030 20:00:17.500896    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 20:00:20.874917 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:20.874884    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 20:00:35.199840 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:35.199807    5907 scope.go:122] "RemoveContainer" containerID="4ce8016ba076c1d6436ba048cbd94144f28fddd306621a53e3a4676fb51f96c9"
Oct 30 20:00:35.208657 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:35.208634    5907 scope.go:122] "RemoveContainer" containerID="373869832d6f1cf3614c31c80a4082c4944493c555fd16e476ee01ecba483a52"
Oct 30 20:00:35.216730 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:35.216710    5907 scope.go:122] "RemoveContainer" containerID="6eb6819a8ddbb33410584fb85212c95325424dd65b145428fffa1b0a4aaeb055"
Oct 30 20:00:38.793780 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:38.793696    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 20:00:47.499674 ip-10-0-132-152 kubenswrapper[5907]: E1030 20:00:47.499630    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 20:00:51.828579 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:51.828539    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-sdj44"]
Oct 30 20:00:51.831112 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:51.828995    5907 cpu_manager.go:414] "containerMap: removing container" podUID="534df7f5-7ced-42ea-b56b-a573ee8ca0c3" containerName="cleanup"
Oct 30 20:00:51.831112 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:51.829008    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="534df7f5-7ced-42ea-b56b-a573ee8ca0c3" containerName="cleanup"
Oct 30 20:00:51.831112 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:51.829108    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="534df7f5-7ced-42ea-b56b-a573ee8ca0c3" containerName="cleanup"
Oct 30 20:00:51.832167 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:51.832152    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-sdj44"
Oct 30 20:00:51.848058 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:51.848030    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:00:51.861280 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:51.861250    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 20:00:51.863615 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:51.863589    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-sdj44"]
Oct 30 20:00:51.982286 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:51.982250    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-452b6d7b-sdj44\" (UID: \"ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-sdj44"
Oct 30 20:00:51.982458 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:51.982420    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87762\" (UniqueName: \"kubernetes.io/projected/ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb-kube-api-access-87762\") pod \"maas-api-revoke-sub-452b6d7b-sdj44\" (UID: \"ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-sdj44"
Oct 30 20:00:51.982507 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:51.982455    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-452b6d7b-sdj44\" (UID: \"ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-sdj44"
Oct 30 20:00:52.083328 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:52.083249    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-452b6d7b-sdj44\" (UID: \"ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-sdj44"
Oct 30 20:00:52.083328 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:52.083308    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-452b6d7b-sdj44\" (UID: \"ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-sdj44"
Oct 30 20:00:52.083589 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:52.083404    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87762\" (UniqueName: \"kubernetes.io/projected/ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb-kube-api-access-87762\") pod \"maas-api-revoke-sub-452b6d7b-sdj44\" (UID: \"ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-sdj44"
Oct 30 20:00:52.083932 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:52.083909    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-452b6d7b-sdj44\" (UID: \"ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-sdj44"
Oct 30 20:00:52.085924 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:52.085893    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-452b6d7b-sdj44\" (UID: \"ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-sdj44"
Oct 30 20:00:52.103908 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:52.103886    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87762\" (UniqueName: \"kubernetes.io/projected/ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb-kube-api-access-87762\") pod \"maas-api-revoke-sub-452b6d7b-sdj44\" (UID: \"ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-sdj44"
Oct 30 20:00:52.145774 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:52.145749    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-sdj44"
Oct 30 20:00:52.446612 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:52.446584    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-sdj44"]
Oct 30 20:00:52.448767 ip-10-0-132-152 kubenswrapper[5907]: W1030 20:00:52.448739    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab7ceb8c_80b8_4791_b02b_1a0930a7dbfb.slice/crio-7e89a5f1b306211602f875ed937b0caecdd14f0581a15cd9f4b8a1d99e2b77f0 WatchSource:0}: Error finding container 7e89a5f1b306211602f875ed937b0caecdd14f0581a15cd9f4b8a1d99e2b77f0: Status 404 returned error can't find the container with id 7e89a5f1b306211602f875ed937b0caecdd14f0581a15cd9f4b8a1d99e2b77f0
Oct 30 20:00:52.966555 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:52.966520    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-sdj44" event={"ID":"ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb","Type":"ContainerStarted","Data":"fe42d33becb85d12799a9b8906b07de001c61ebdfd0d90e325d1b7b2e8d77ae3"}
Oct 30 20:00:52.966983 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:52.966574    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-sdj44" event={"ID":"ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb","Type":"ContainerStarted","Data":"7e89a5f1b306211602f875ed937b0caecdd14f0581a15cd9f4b8a1d99e2b77f0"}
Oct 30 20:00:53.007897 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:53.007757    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-sdj44" podStartSLOduration=2.007738883 podStartE2EDuration="2.007738883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:00:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:00:53.007256072 +0000 UTC m=+2364.850188011" watchObservedRunningTime="2026-10-30 20:00:53.007738883 +0000 UTC m=+2364.850670825"
Oct 30 20:00:53.155150 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:53.155113    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 20:00:53.979012 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:53.978973    5907 generic.go:356] "Generic (PLEG): container finished" podID="ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb" containerID="fe42d33becb85d12799a9b8906b07de001c61ebdfd0d90e325d1b7b2e8d77ae3" exitCode=0
Oct 30 20:00:53.979563 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:53.979067    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-sdj44" event={"ID":"ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb","Type":"ContainerDied","Data":"fe42d33becb85d12799a9b8906b07de001c61ebdfd0d90e325d1b7b2e8d77ae3"}
Oct 30 20:00:55.863888 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:55.853133    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-sdj44"
Oct 30 20:00:55.922916 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:55.911266    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb-maas-api-xks-ca\") pod \"ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb\" (UID: \"ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb\") "
Oct 30 20:00:55.922916 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:55.911345    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb-maas-api-openshift-ca\") pod \"ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb\" (UID: \"ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb\") "
Oct 30 20:00:55.922916 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:55.911416    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb-kube-api-access-87762\" (UniqueName: \"kubernetes.io/projected/ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb-kube-api-access-87762\") pod \"ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb\" (UID: \"ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb\") "
Oct 30 20:00:55.922916 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:55.917083    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb-maas-api-openshift-ca" pod "ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb" (UID: "ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:00:55.926891 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:55.926610    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb-kube-api-access-87762" pod "ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb" (UID: "ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb"). InnerVolumeSpecName "kube-api-access-87762". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:00:55.963887 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:55.946202    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb-maas-api-xks-ca" pod "ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb" (UID: "ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:00:56.026964 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:56.026536    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 20:00:56.026964 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:56.026575    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 20:00:56.026964 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:56.026594    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87762\" (UniqueName: \"kubernetes.io/projected/ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb-kube-api-access-87762\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 20:00:56.133886 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:56.110540    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-sdj44" event={"ID":"ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb","Type":"ContainerDied","Data":"7e89a5f1b306211602f875ed937b0caecdd14f0581a15cd9f4b8a1d99e2b77f0"}
Oct 30 20:00:56.133886 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:56.110615    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7e89a5f1b306211602f875ed937b0caecdd14f0581a15cd9f4b8a1d99e2b77f0"
Oct 30 20:00:56.133886 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:00:56.110697    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-sdj44"
Oct 30 20:01:04.874819 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:04.874780    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 20:01:07.042016 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:07.041984    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a45e4b2b-k7h6z"]
Oct 30 20:01:07.051891 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:07.046955    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a45e4b2b-k7h6z"]
Oct 30 20:01:08.333676 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:08.327201    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1ee0f668-16b6-43ca-b77d-c16bf3cd6347" path="/var/lib/kubelet/pods/1ee0f668-16b6-43ca-b77d-c16bf3cd6347/volumes"
Oct 30 20:01:10.934517 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:10.934478    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2brf5"]
Oct 30 20:01:10.939146 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:10.935293    5907 cpu_manager.go:414] "containerMap: removing container" podUID="ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb" containerName="revoke-keys"
Oct 30 20:01:10.939146 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:10.935315    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb" containerName="revoke-keys"
Oct 30 20:01:10.939146 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:10.935372    5907 cpu_manager.go:414] "containerMap: removing container" podUID="534df7f5-7ced-42ea-b56b-a573ee8ca0c3" containerName="cleanup"
Oct 30 20:01:10.939146 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:10.935382    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="534df7f5-7ced-42ea-b56b-a573ee8ca0c3" containerName="cleanup"
Oct 30 20:01:10.939146 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:10.935523    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="534df7f5-7ced-42ea-b56b-a573ee8ca0c3" containerName="cleanup"
Oct 30 20:01:10.939146 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:10.935545    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb" containerName="revoke-keys"
Oct 30 20:01:10.941566 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:10.941527    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2brf5"
Oct 30 20:01:10.963047 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:10.963019    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2brf5"]
Oct 30 20:01:10.973429 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:10.973398    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:01:10.973595 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:10.973439    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 20:01:11.092013 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:11.091956    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/761a9956-10ee-4642-948d-3c34f238daa0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3e31a506-2brf5\" (UID: \"761a9956-10ee-4642-948d-3c34f238daa0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2brf5"
Oct 30 20:01:11.092232 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:11.092130    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/761a9956-10ee-4642-948d-3c34f238daa0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3e31a506-2brf5\" (UID: \"761a9956-10ee-4642-948d-3c34f238daa0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2brf5"
Oct 30 20:01:11.092232 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:11.092191    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc669\" (UniqueName: \"kubernetes.io/projected/761a9956-10ee-4642-948d-3c34f238daa0-kube-api-access-nc669\") pod \"maas-api-revoke-sub-3e31a506-2brf5\" (UID: \"761a9956-10ee-4642-948d-3c34f238daa0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2brf5"
Oct 30 20:01:11.193171 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:11.193067    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/761a9956-10ee-4642-948d-3c34f238daa0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3e31a506-2brf5\" (UID: \"761a9956-10ee-4642-948d-3c34f238daa0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2brf5"
Oct 30 20:01:11.193359 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:11.193179    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/761a9956-10ee-4642-948d-3c34f238daa0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3e31a506-2brf5\" (UID: \"761a9956-10ee-4642-948d-3c34f238daa0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2brf5"
Oct 30 20:01:11.193359 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:11.193232    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nc669\" (UniqueName: \"kubernetes.io/projected/761a9956-10ee-4642-948d-3c34f238daa0-kube-api-access-nc669\") pod \"maas-api-revoke-sub-3e31a506-2brf5\" (UID: \"761a9956-10ee-4642-948d-3c34f238daa0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2brf5"
Oct 30 20:01:11.194218 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:11.194183    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/761a9956-10ee-4642-948d-3c34f238daa0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3e31a506-2brf5\" (UID: \"761a9956-10ee-4642-948d-3c34f238daa0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2brf5"
Oct 30 20:01:11.196368 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:11.196341    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/761a9956-10ee-4642-948d-3c34f238daa0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3e31a506-2brf5\" (UID: \"761a9956-10ee-4642-948d-3c34f238daa0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2brf5"
Oct 30 20:01:11.216005 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:11.215977    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc669\" (UniqueName: \"kubernetes.io/projected/761a9956-10ee-4642-948d-3c34f238daa0-kube-api-access-nc669\") pod \"maas-api-revoke-sub-3e31a506-2brf5\" (UID: \"761a9956-10ee-4642-948d-3c34f238daa0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2brf5"
Oct 30 20:01:11.263737 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:11.263696    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2brf5"
Oct 30 20:01:12.001906 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:11.996502    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2brf5"]
Oct 30 20:01:12.034885 ip-10-0-132-152 kubenswrapper[5907]: W1030 20:01:12.033009    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod761a9956_10ee_4642_948d_3c34f238daa0.slice/crio-2ca1350ce08ae9ac1f3d03c48b851aa861dda75a3f8416ec93615328aebe9958 WatchSource:0}: Error finding container 2ca1350ce08ae9ac1f3d03c48b851aa861dda75a3f8416ec93615328aebe9958: Status 404 returned error can't find the container with id 2ca1350ce08ae9ac1f3d03c48b851aa861dda75a3f8416ec93615328aebe9958
Oct 30 20:01:12.261937 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:12.261684    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2brf5" event={"ID":"761a9956-10ee-4642-948d-3c34f238daa0","Type":"ContainerStarted","Data":"2ca1350ce08ae9ac1f3d03c48b851aa861dda75a3f8416ec93615328aebe9958"}
Oct 30 20:01:13.268628 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:13.268599    5907 generic.go:356] "Generic (PLEG): container finished" podID="761a9956-10ee-4642-948d-3c34f238daa0" containerID="93e6cd29d573b4921d7276d615b97d92d3d9f1edfae8123382d5067042ba170e" exitCode=0
Oct 30 20:01:13.268995 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:13.268681    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2brf5" event={"ID":"761a9956-10ee-4642-948d-3c34f238daa0","Type":"ContainerDied","Data":"93e6cd29d573b4921d7276d615b97d92d3d9f1edfae8123382d5067042ba170e"}
Oct 30 20:01:13.877206 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:13.877171    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 20:01:14.407928 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:14.407902    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2brf5"
Oct 30 20:01:14.513854 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:14.513820    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/761a9956-10ee-4642-948d-3c34f238daa0-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/761a9956-10ee-4642-948d-3c34f238daa0-maas-api-xks-ca\") pod \"761a9956-10ee-4642-948d-3c34f238daa0\" (UID: \"761a9956-10ee-4642-948d-3c34f238daa0\") "
Oct 30 20:01:14.514062 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:14.513887    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/761a9956-10ee-4642-948d-3c34f238daa0-kube-api-access-nc669\" (UniqueName: \"kubernetes.io/projected/761a9956-10ee-4642-948d-3c34f238daa0-kube-api-access-nc669\") pod \"761a9956-10ee-4642-948d-3c34f238daa0\" (UID: \"761a9956-10ee-4642-948d-3c34f238daa0\") "
Oct 30 20:01:14.514062 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:14.513942    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/761a9956-10ee-4642-948d-3c34f238daa0-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/761a9956-10ee-4642-948d-3c34f238daa0-maas-api-openshift-ca\") pod \"761a9956-10ee-4642-948d-3c34f238daa0\" (UID: \"761a9956-10ee-4642-948d-3c34f238daa0\") "
Oct 30 20:01:14.514361 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:14.514331    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/761a9956-10ee-4642-948d-3c34f238daa0-maas-api-openshift-ca" pod "761a9956-10ee-4642-948d-3c34f238daa0" (UID: "761a9956-10ee-4642-948d-3c34f238daa0"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:01:14.514483 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:14.514457    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/761a9956-10ee-4642-948d-3c34f238daa0-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 20:01:14.516063 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:14.516040    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/761a9956-10ee-4642-948d-3c34f238daa0-maas-api-xks-ca" pod "761a9956-10ee-4642-948d-3c34f238daa0" (UID: "761a9956-10ee-4642-948d-3c34f238daa0"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:01:14.516190 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:14.516172    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/761a9956-10ee-4642-948d-3c34f238daa0-kube-api-access-nc669" pod "761a9956-10ee-4642-948d-3c34f238daa0" (UID: "761a9956-10ee-4642-948d-3c34f238daa0"). InnerVolumeSpecName "kube-api-access-nc669". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:01:14.615336 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:14.615309    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/761a9956-10ee-4642-948d-3c34f238daa0-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 20:01:14.615336 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:14.615334    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nc669\" (UniqueName: \"kubernetes.io/projected/761a9956-10ee-4642-948d-3c34f238daa0-kube-api-access-nc669\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 20:01:15.280900 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:15.280845    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2brf5"
Oct 30 20:01:15.280900 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:15.280881    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2brf5" event={"ID":"761a9956-10ee-4642-948d-3c34f238daa0","Type":"ContainerDied","Data":"2ca1350ce08ae9ac1f3d03c48b851aa861dda75a3f8416ec93615328aebe9958"}
Oct 30 20:01:15.281143 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:15.280928    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2ca1350ce08ae9ac1f3d03c48b851aa861dda75a3f8416ec93615328aebe9958"
Oct 30 20:01:17.501756 ip-10-0-132-152 kubenswrapper[5907]: E1030 20:01:17.501710    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 20:01:22.172690 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:22.172654    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 20:01:29.391918 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:29.391882    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 20:01:34.429503 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:34.429466    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2067b2c-9jjqm"]
Oct 30 20:01:34.429956 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:34.429941    5907 cpu_manager.go:414] "containerMap: removing container" podUID="761a9956-10ee-4642-948d-3c34f238daa0" containerName="revoke-keys"
Oct 30 20:01:34.430015 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:34.429956    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="761a9956-10ee-4642-948d-3c34f238daa0" containerName="revoke-keys"
Oct 30 20:01:34.430055 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:34.430039    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="761a9956-10ee-4642-948d-3c34f238daa0" containerName="revoke-keys"
Oct 30 20:01:34.434396 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:34.434370    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2067b2c-9jjqm"
Oct 30 20:01:34.454654 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:34.454621    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2067b2c-9jjqm"]
Oct 30 20:01:34.464786 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:34.464756    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:01:34.464968 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:34.464840    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q4qb5\""
Oct 30 20:01:34.499835 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:34.499800    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/744e211a-0530-4633-9629-d0d740e4684d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d2067b2c-9jjqm\" (UID: \"744e211a-0530-4633-9629-d0d740e4684d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2067b2c-9jjqm"
Oct 30 20:01:34.500028 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:34.499841    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qzdv\" (UniqueName: \"kubernetes.io/projected/744e211a-0530-4633-9629-d0d740e4684d-kube-api-access-2qzdv\") pod \"maas-api-revoke-sub-d2067b2c-9jjqm\" (UID: \"744e211a-0530-4633-9629-d0d740e4684d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2067b2c-9jjqm"
Oct 30 20:01:34.500088 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:34.500063    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/744e211a-0530-4633-9629-d0d740e4684d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d2067b2c-9jjqm\" (UID: \"744e211a-0530-4633-9629-d0d740e4684d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2067b2c-9jjqm"
Oct 30 20:01:34.600939 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:34.600901    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/744e211a-0530-4633-9629-d0d740e4684d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d2067b2c-9jjqm\" (UID: \"744e211a-0530-4633-9629-d0d740e4684d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2067b2c-9jjqm"
Oct 30 20:01:34.601143 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:34.601033    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/744e211a-0530-4633-9629-d0d740e4684d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d2067b2c-9jjqm\" (UID: \"744e211a-0530-4633-9629-d0d740e4684d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2067b2c-9jjqm"
Oct 30 20:01:34.601143 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:34.601074    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qzdv\" (UniqueName: \"kubernetes.io/projected/744e211a-0530-4633-9629-d0d740e4684d-kube-api-access-2qzdv\") pod \"maas-api-revoke-sub-d2067b2c-9jjqm\" (UID: \"744e211a-0530-4633-9629-d0d740e4684d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2067b2c-9jjqm"
Oct 30 20:01:34.601605 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:34.601579    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/744e211a-0530-4633-9629-d0d740e4684d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d2067b2c-9jjqm\" (UID: \"744e211a-0530-4633-9629-d0d740e4684d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2067b2c-9jjqm"
Oct 30 20:01:34.603473 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:34.603455    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/744e211a-0530-4633-9629-d0d740e4684d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d2067b2c-9jjqm\" (UID: \"744e211a-0530-4633-9629-d0d740e4684d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2067b2c-9jjqm"
Oct 30 20:01:34.621940 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:34.621916    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qzdv\" (UniqueName: \"kubernetes.io/projected/744e211a-0530-4633-9629-d0d740e4684d-kube-api-access-2qzdv\") pod \"maas-api-revoke-sub-d2067b2c-9jjqm\" (UID: \"744e211a-0530-4633-9629-d0d740e4684d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2067b2c-9jjqm"
Oct 30 20:01:34.749523 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:34.749436    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2067b2c-9jjqm"
Oct 30 20:01:35.050517 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:35.050406    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2067b2c-9jjqm"]
Oct 30 20:01:35.053016 ip-10-0-132-152 kubenswrapper[5907]: W1030 20:01:35.052981    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod744e211a_0530_4633_9629_d0d740e4684d.slice/crio-878f0591e56cb9b752a9ce40137962c0670cc7e322eda90ebd0ed27ccf999bf9 WatchSource:0}: Error finding container 878f0591e56cb9b752a9ce40137962c0670cc7e322eda90ebd0ed27ccf999bf9: Status 404 returned error can't find the container with id 878f0591e56cb9b752a9ce40137962c0670cc7e322eda90ebd0ed27ccf999bf9
Oct 30 20:01:35.248147 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:35.248116    5907 scope.go:122] "RemoveContainer" containerID="dd74af0fcd1ffd1dea6c17995b8169d9e798a1e86e2b0b3def8598440f5cbbca"
Oct 30 20:01:35.256619 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:35.256599    5907 scope.go:122] "RemoveContainer" containerID="2832ffbb9a2bf7ad03c253d476f988297f81745120aa3a87e06d75cb1a3d1366"
Oct 30 20:01:35.445795 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:35.445759    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2067b2c-9jjqm" event={"ID":"744e211a-0530-4633-9629-d0d740e4684d","Type":"ContainerStarted","Data":"72a8cc1a33bac52f4d546e0e5c768e71b9e2e21fb36512d5401b84d000338a70"}
Oct 30 20:01:35.446205 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:35.445813    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2067b2c-9jjqm" event={"ID":"744e211a-0530-4633-9629-d0d740e4684d","Type":"ContainerStarted","Data":"878f0591e56cb9b752a9ce40137962c0670cc7e322eda90ebd0ed27ccf999bf9"}
Oct 30 20:01:35.488179 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:35.488127    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2067b2c-9jjqm" podStartSLOduration=1.488115244 podStartE2EDuration="1.488115244s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:01:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:01:35.486315168 +0000 UTC m=+2407.329247104" watchObservedRunningTime="2026-10-30 20:01:35.488115244 +0000 UTC m=+2407.331047182"
Oct 30 20:01:36.452680 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:36.452639    5907 generic.go:356] "Generic (PLEG): container finished" podID="744e211a-0530-4633-9629-d0d740e4684d" containerID="72a8cc1a33bac52f4d546e0e5c768e71b9e2e21fb36512d5401b84d000338a70" exitCode=0
Oct 30 20:01:36.453118 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:36.452696    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2067b2c-9jjqm" event={"ID":"744e211a-0530-4633-9629-d0d740e4684d","Type":"ContainerDied","Data":"72a8cc1a33bac52f4d546e0e5c768e71b9e2e21fb36512d5401b84d000338a70"}
Oct 30 20:01:37.599268 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:37.599241    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2067b2c-9jjqm"
Oct 30 20:01:37.735420 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:37.735339    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/744e211a-0530-4633-9629-d0d740e4684d-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/744e211a-0530-4633-9629-d0d740e4684d-maas-api-openshift-ca\") pod \"744e211a-0530-4633-9629-d0d740e4684d\" (UID: \"744e211a-0530-4633-9629-d0d740e4684d\") "
Oct 30 20:01:37.735420 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:37.735379    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/744e211a-0530-4633-9629-d0d740e4684d-kube-api-access-2qzdv\" (UniqueName: \"kubernetes.io/projected/744e211a-0530-4633-9629-d0d740e4684d-kube-api-access-2qzdv\") pod \"744e211a-0530-4633-9629-d0d740e4684d\" (UID: \"744e211a-0530-4633-9629-d0d740e4684d\") "
Oct 30 20:01:37.735676 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:37.735444    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/744e211a-0530-4633-9629-d0d740e4684d-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/744e211a-0530-4633-9629-d0d740e4684d-maas-api-xks-ca\") pod \"744e211a-0530-4633-9629-d0d740e4684d\" (UID: \"744e211a-0530-4633-9629-d0d740e4684d\") "
Oct 30 20:01:37.735752 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:37.735702    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/744e211a-0530-4633-9629-d0d740e4684d-maas-api-openshift-ca" pod "744e211a-0530-4633-9629-d0d740e4684d" (UID: "744e211a-0530-4633-9629-d0d740e4684d"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:01:37.737699 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:37.737673    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/744e211a-0530-4633-9629-d0d740e4684d-maas-api-xks-ca" pod "744e211a-0530-4633-9629-d0d740e4684d" (UID: "744e211a-0530-4633-9629-d0d740e4684d"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:01:37.737699 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:37.737681    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/744e211a-0530-4633-9629-d0d740e4684d-kube-api-access-2qzdv" pod "744e211a-0530-4633-9629-d0d740e4684d" (UID: "744e211a-0530-4633-9629-d0d740e4684d"). InnerVolumeSpecName "kube-api-access-2qzdv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:01:37.837030 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:37.836999    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/744e211a-0530-4633-9629-d0d740e4684d-maas-api-xks-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 20:01:37.837030 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:37.837024    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/744e211a-0530-4633-9629-d0d740e4684d-maas-api-openshift-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 20:01:37.837030 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:37.837035    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qzdv\" (UniqueName: \"kubernetes.io/projected/744e211a-0530-4633-9629-d0d740e4684d-kube-api-access-2qzdv\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 20:01:38.464297 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:38.464259    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2067b2c-9jjqm" event={"ID":"744e211a-0530-4633-9629-d0d740e4684d","Type":"ContainerDied","Data":"878f0591e56cb9b752a9ce40137962c0670cc7e322eda90ebd0ed27ccf999bf9"}
Oct 30 20:01:38.464708 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:38.464306    5907 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="878f0591e56cb9b752a9ce40137962c0670cc7e322eda90ebd0ed27ccf999bf9"
Oct 30 20:01:38.464708 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:38.464266    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2067b2c-9jjqm"
Oct 30 20:01:39.031301 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:39.031264    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-smhtk"]
Oct 30 20:01:39.044299 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:39.044271    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-smhtk"]
Oct 30 20:01:40.286886 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:40.286821    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e12a469d-c556-4761-a182-8c5498d7b180" path="/var/lib/kubelet/pods/e12a469d-c556-4761-a182-8c5498d7b180/volumes"
Oct 30 20:01:45.793526 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:45.793493    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 20:01:46.909969 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:46.909937    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5f5c657c85-jtpzc"]
Oct 30 20:01:46.910443 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:46.910419    5907 cpu_manager.go:414] "containerMap: removing container" podUID="744e211a-0530-4633-9629-d0d740e4684d" containerName="revoke-keys"
Oct 30 20:01:46.910443 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:46.910432    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="744e211a-0530-4633-9629-d0d740e4684d" containerName="revoke-keys"
Oct 30 20:01:46.910538 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:46.910504    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="744e211a-0530-4633-9629-d0d740e4684d" containerName="revoke-keys"
Oct 30 20:01:46.913801 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:46.913781    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5f5c657c85-jtpzc"
Oct 30 20:01:46.928916 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:46.928894    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76b7c060-0a4a-4edb-a9c3-c89badeb1285-openshift-service-ca\") pod \"maas-api-5f5c657c85-jtpzc\" (UID: \"76b7c060-0a4a-4edb-a9c3-c89badeb1285\") " pod="odh-ai-gateway-infra/maas-api-5f5c657c85-jtpzc"
Oct 30 20:01:46.929040 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:46.928945    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76b7c060-0a4a-4edb-a9c3-c89badeb1285-maas-api-tls\") pod \"maas-api-5f5c657c85-jtpzc\" (UID: \"76b7c060-0a4a-4edb-a9c3-c89badeb1285\") " pod="odh-ai-gateway-infra/maas-api-5f5c657c85-jtpzc"
Oct 30 20:01:46.929084 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:46.929032    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smxzq\" (UniqueName: \"kubernetes.io/projected/76b7c060-0a4a-4edb-a9c3-c89badeb1285-kube-api-access-smxzq\") pod \"maas-api-5f5c657c85-jtpzc\" (UID: \"76b7c060-0a4a-4edb-a9c3-c89badeb1285\") " pod="odh-ai-gateway-infra/maas-api-5f5c657c85-jtpzc"
Oct 30 20:01:46.929121 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:46.929091    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/76b7c060-0a4a-4edb-a9c3-c89badeb1285-metrics-certs\") pod \"maas-api-5f5c657c85-jtpzc\" (UID: \"76b7c060-0a4a-4edb-a9c3-c89badeb1285\") " pod="odh-ai-gateway-infra/maas-api-5f5c657c85-jtpzc"
Oct 30 20:01:46.955550 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:46.955521    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5f5c657c85-jtpzc"]
Oct 30 20:01:46.986879 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:46.986835    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Oct 30 20:01:46.987076 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:46.986931    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-g97rg\""
Oct 30 20:01:46.987076 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:46.986972    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 20:01:47.030416 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:47.030373    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/76b7c060-0a4a-4edb-a9c3-c89badeb1285-metrics-certs\") pod \"maas-api-5f5c657c85-jtpzc\" (UID: \"76b7c060-0a4a-4edb-a9c3-c89badeb1285\") " pod="odh-ai-gateway-infra/maas-api-5f5c657c85-jtpzc"
Oct 30 20:01:47.030596 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:47.030464    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76b7c060-0a4a-4edb-a9c3-c89badeb1285-openshift-service-ca\") pod \"maas-api-5f5c657c85-jtpzc\" (UID: \"76b7c060-0a4a-4edb-a9c3-c89badeb1285\") " pod="odh-ai-gateway-infra/maas-api-5f5c657c85-jtpzc"
Oct 30 20:01:47.030596 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:47.030525    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76b7c060-0a4a-4edb-a9c3-c89badeb1285-maas-api-tls\") pod \"maas-api-5f5c657c85-jtpzc\" (UID: \"76b7c060-0a4a-4edb-a9c3-c89badeb1285\") " pod="odh-ai-gateway-infra/maas-api-5f5c657c85-jtpzc"
Oct 30 20:01:47.030596 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:47.030578    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smxzq\" (UniqueName: \"kubernetes.io/projected/76b7c060-0a4a-4edb-a9c3-c89badeb1285-kube-api-access-smxzq\") pod \"maas-api-5f5c657c85-jtpzc\" (UID: \"76b7c060-0a4a-4edb-a9c3-c89badeb1285\") " pod="odh-ai-gateway-infra/maas-api-5f5c657c85-jtpzc"
Oct 30 20:01:47.031213 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:47.031187    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76b7c060-0a4a-4edb-a9c3-c89badeb1285-openshift-service-ca\") pod \"maas-api-5f5c657c85-jtpzc\" (UID: \"76b7c060-0a4a-4edb-a9c3-c89badeb1285\") " pod="odh-ai-gateway-infra/maas-api-5f5c657c85-jtpzc"
Oct 30 20:01:47.033265 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:47.033240    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76b7c060-0a4a-4edb-a9c3-c89badeb1285-maas-api-tls\") pod \"maas-api-5f5c657c85-jtpzc\" (UID: \"76b7c060-0a4a-4edb-a9c3-c89badeb1285\") " pod="odh-ai-gateway-infra/maas-api-5f5c657c85-jtpzc"
Oct 30 20:01:47.033352 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:47.033246    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/76b7c060-0a4a-4edb-a9c3-c89badeb1285-metrics-certs\") pod \"maas-api-5f5c657c85-jtpzc\" (UID: \"76b7c060-0a4a-4edb-a9c3-c89badeb1285\") " pod="odh-ai-gateway-infra/maas-api-5f5c657c85-jtpzc"
Oct 30 20:01:47.051230 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:47.051204    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-smxzq\" (UniqueName: \"kubernetes.io/projected/76b7c060-0a4a-4edb-a9c3-c89badeb1285-kube-api-access-smxzq\") pod \"maas-api-5f5c657c85-jtpzc\" (UID: \"76b7c060-0a4a-4edb-a9c3-c89badeb1285\") " pod="odh-ai-gateway-infra/maas-api-5f5c657c85-jtpzc"
Oct 30 20:01:47.226626 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:47.226537    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5f5c657c85-jtpzc"
Oct 30 20:01:47.501888 ip-10-0-132-152 kubenswrapper[5907]: E1030 20:01:47.501784    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 20:01:47.530124 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:47.530096    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5f5c657c85-jtpzc"]
Oct 30 20:01:47.533277 ip-10-0-132-152 kubenswrapper[5907]: W1030 20:01:47.533242    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76b7c060_0a4a_4edb_a9c3_c89badeb1285.slice/crio-ee736df6d8005115da49890bf1ba400a2fae9dac25c67ccfb6c9a4f8df65c5d0 WatchSource:0}: Error finding container ee736df6d8005115da49890bf1ba400a2fae9dac25c67ccfb6c9a4f8df65c5d0: Status 404 returned error can't find the container with id ee736df6d8005115da49890bf1ba400a2fae9dac25c67ccfb6c9a4f8df65c5d0
Oct 30 20:01:47.535004 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:47.534985    5907 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 20:01:48.515937 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:48.515891    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5f5c657c85-jtpzc" event={"ID":"76b7c060-0a4a-4edb-a9c3-c89badeb1285","Type":"ContainerStarted","Data":"bac1b08951dc397623ac34e9b2c8cd7dcc799df3b6b82bf22076e811e538e54b"}
Oct 30 20:01:48.516344 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:48.515958    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5f5c657c85-jtpzc" event={"ID":"76b7c060-0a4a-4edb-a9c3-c89badeb1285","Type":"ContainerStarted","Data":"ee736df6d8005115da49890bf1ba400a2fae9dac25c67ccfb6c9a4f8df65c5d0"}
Oct 30 20:01:48.516344 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:48.516092    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5f5c657c85-jtpzc"
Oct 30 20:01:48.558613 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:48.558563    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5f5c657c85-jtpzc" podStartSLOduration=2.144783584 podStartE2EDuration="2.558549364s" totalImagesPullingTime="413.76578ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:01:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:01:48.55666978 +0000 UTC m=+2420.399601718" watchObservedRunningTime="2026-10-30 20:01:48.558549364 +0000 UTC m=+2420.401481346"
Oct 30 20:01:54.529282 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:54.529249    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5f5c657c85-jtpzc"
Oct 30 20:01:54.679143 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:54.679113    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-9dcd87b65-p9k2r"]
Oct 30 20:01:54.682698 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:54.682680    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-9dcd87b65-p9k2r"
Oct 30 20:01:54.705936 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:54.705908    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-9dcd87b65-p9k2r"]
Oct 30 20:01:54.707853 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:54.707829    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zw226\" (UniqueName: \"kubernetes.io/projected/416b5aec-c0a5-48ea-9716-7e23bd98192c-kube-api-access-zw226\") pod \"maas-api-9dcd87b65-p9k2r\" (UID: \"416b5aec-c0a5-48ea-9716-7e23bd98192c\") " pod="odh-ai-gateway-infra/maas-api-9dcd87b65-p9k2r"
Oct 30 20:01:54.708005 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:54.707960    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/416b5aec-c0a5-48ea-9716-7e23bd98192c-openshift-service-ca\") pod \"maas-api-9dcd87b65-p9k2r\" (UID: \"416b5aec-c0a5-48ea-9716-7e23bd98192c\") " pod="odh-ai-gateway-infra/maas-api-9dcd87b65-p9k2r"
Oct 30 20:01:54.708063 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:54.708040    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/416b5aec-c0a5-48ea-9716-7e23bd98192c-maas-api-tls\") pod \"maas-api-9dcd87b65-p9k2r\" (UID: \"416b5aec-c0a5-48ea-9716-7e23bd98192c\") " pod="odh-ai-gateway-infra/maas-api-9dcd87b65-p9k2r"
Oct 30 20:01:54.708124 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:54.708110    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/416b5aec-c0a5-48ea-9716-7e23bd98192c-metrics-certs\") pod \"maas-api-9dcd87b65-p9k2r\" (UID: \"416b5aec-c0a5-48ea-9716-7e23bd98192c\") " pod="odh-ai-gateway-infra/maas-api-9dcd87b65-p9k2r"
Oct 30 20:01:54.809678 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:54.809595    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/416b5aec-c0a5-48ea-9716-7e23bd98192c-maas-api-tls\") pod \"maas-api-9dcd87b65-p9k2r\" (UID: \"416b5aec-c0a5-48ea-9716-7e23bd98192c\") " pod="odh-ai-gateway-infra/maas-api-9dcd87b65-p9k2r"
Oct 30 20:01:54.809678 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:54.809654    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/416b5aec-c0a5-48ea-9716-7e23bd98192c-metrics-certs\") pod \"maas-api-9dcd87b65-p9k2r\" (UID: \"416b5aec-c0a5-48ea-9716-7e23bd98192c\") " pod="odh-ai-gateway-infra/maas-api-9dcd87b65-p9k2r"
Oct 30 20:01:54.809927 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:54.809701    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zw226\" (UniqueName: \"kubernetes.io/projected/416b5aec-c0a5-48ea-9716-7e23bd98192c-kube-api-access-zw226\") pod \"maas-api-9dcd87b65-p9k2r\" (UID: \"416b5aec-c0a5-48ea-9716-7e23bd98192c\") " pod="odh-ai-gateway-infra/maas-api-9dcd87b65-p9k2r"
Oct 30 20:01:54.809927 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:54.809744    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/416b5aec-c0a5-48ea-9716-7e23bd98192c-openshift-service-ca\") pod \"maas-api-9dcd87b65-p9k2r\" (UID: \"416b5aec-c0a5-48ea-9716-7e23bd98192c\") " pod="odh-ai-gateway-infra/maas-api-9dcd87b65-p9k2r"
Oct 30 20:01:54.810435 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:54.810405    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/416b5aec-c0a5-48ea-9716-7e23bd98192c-openshift-service-ca\") pod \"maas-api-9dcd87b65-p9k2r\" (UID: \"416b5aec-c0a5-48ea-9716-7e23bd98192c\") " pod="odh-ai-gateway-infra/maas-api-9dcd87b65-p9k2r"
Oct 30 20:01:54.812168 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:54.812144    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/416b5aec-c0a5-48ea-9716-7e23bd98192c-maas-api-tls\") pod \"maas-api-9dcd87b65-p9k2r\" (UID: \"416b5aec-c0a5-48ea-9716-7e23bd98192c\") " pod="odh-ai-gateway-infra/maas-api-9dcd87b65-p9k2r"
Oct 30 20:01:54.812320 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:54.812297    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/416b5aec-c0a5-48ea-9716-7e23bd98192c-metrics-certs\") pod \"maas-api-9dcd87b65-p9k2r\" (UID: \"416b5aec-c0a5-48ea-9716-7e23bd98192c\") " pod="odh-ai-gateway-infra/maas-api-9dcd87b65-p9k2r"
Oct 30 20:01:54.834347 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:54.834320    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zw226\" (UniqueName: \"kubernetes.io/projected/416b5aec-c0a5-48ea-9716-7e23bd98192c-kube-api-access-zw226\") pod \"maas-api-9dcd87b65-p9k2r\" (UID: \"416b5aec-c0a5-48ea-9716-7e23bd98192c\") " pod="odh-ai-gateway-infra/maas-api-9dcd87b65-p9k2r"
Oct 30 20:01:54.997201 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:54.997168    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-9dcd87b65-p9k2r"
Oct 30 20:01:55.302222 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:55.302197    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-9dcd87b65-p9k2r"]
Oct 30 20:01:55.304606 ip-10-0-132-152 kubenswrapper[5907]: W1030 20:01:55.304582    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod416b5aec_c0a5_48ea_9716_7e23bd98192c.slice/crio-be2a40cc67d2c26c0cd1ad98a9660bcc74b2421ad5893d80093b06e05280b711 WatchSource:0}: Error finding container be2a40cc67d2c26c0cd1ad98a9660bcc74b2421ad5893d80093b06e05280b711: Status 404 returned error can't find the container with id be2a40cc67d2c26c0cd1ad98a9660bcc74b2421ad5893d80093b06e05280b711
Oct 30 20:01:55.553683 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:55.553593    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-9dcd87b65-p9k2r" event={"ID":"416b5aec-c0a5-48ea-9716-7e23bd98192c","Type":"ContainerStarted","Data":"be2a40cc67d2c26c0cd1ad98a9660bcc74b2421ad5893d80093b06e05280b711"}
Oct 30 20:01:56.560938 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:56.560894    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-9dcd87b65-p9k2r" event={"ID":"416b5aec-c0a5-48ea-9716-7e23bd98192c","Type":"ContainerStarted","Data":"aafa1389856c5e55deac5dcc0c5ab3555e7549436d07d63d82dd29210f7d8c29"}
Oct 30 20:01:56.561374 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:56.561072    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-9dcd87b65-p9k2r"
Oct 30 20:01:56.607277 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:01:56.607225    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-9dcd87b65-p9k2r" podStartSLOduration=2.3068672230000002 podStartE2EDuration="2.607212203s" totalImagesPullingTime="300.34498ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:01:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:01:56.604482209 +0000 UTC m=+2428.447414147" watchObservedRunningTime="2026-10-30 20:01:56.607212203 +0000 UTC m=+2428.450144138"
Oct 30 20:02:02.576662 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:02:02.576626    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-9dcd87b65-p9k2r"
Oct 30 20:02:02.679933 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:02:02.679899    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5f5c657c85-jtpzc"]
Oct 30 20:02:02.680134 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:02:02.680113    5907 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5f5c657c85-jtpzc" podUID="76b7c060-0a4a-4edb-a9c3-c89badeb1285" containerName="maas-api" containerID="cri-o://bac1b08951dc397623ac34e9b2c8cd7dcc799df3b6b82bf22076e811e538e54b" gracePeriod=30
Oct 30 20:02:02.945780 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:02:02.945755    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5f5c657c85-jtpzc"
Oct 30 20:02:03.002329 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:02:03.002296    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/76b7c060-0a4a-4edb-a9c3-c89badeb1285-kube-api-access-smxzq\" (UniqueName: \"kubernetes.io/projected/76b7c060-0a4a-4edb-a9c3-c89badeb1285-kube-api-access-smxzq\") pod \"76b7c060-0a4a-4edb-a9c3-c89badeb1285\" (UID: \"76b7c060-0a4a-4edb-a9c3-c89badeb1285\") "
Oct 30 20:02:03.002518 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:02:03.002346    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/76b7c060-0a4a-4edb-a9c3-c89badeb1285-metrics-certs\" (UniqueName: \"kubernetes.io/secret/76b7c060-0a4a-4edb-a9c3-c89badeb1285-metrics-certs\") pod \"76b7c060-0a4a-4edb-a9c3-c89badeb1285\" (UID: \"76b7c060-0a4a-4edb-a9c3-c89badeb1285\") "
Oct 30 20:02:03.002518 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:02:03.002409    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/76b7c060-0a4a-4edb-a9c3-c89badeb1285-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76b7c060-0a4a-4edb-a9c3-c89badeb1285-openshift-service-ca\") pod \"76b7c060-0a4a-4edb-a9c3-c89badeb1285\" (UID: \"76b7c060-0a4a-4edb-a9c3-c89badeb1285\") "
Oct 30 20:02:03.002518 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:02:03.002467    5907 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/76b7c060-0a4a-4edb-a9c3-c89badeb1285-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76b7c060-0a4a-4edb-a9c3-c89badeb1285-maas-api-tls\") pod \"76b7c060-0a4a-4edb-a9c3-c89badeb1285\" (UID: \"76b7c060-0a4a-4edb-a9c3-c89badeb1285\") "
Oct 30 20:02:03.002839 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:02:03.002811    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76b7c060-0a4a-4edb-a9c3-c89badeb1285-openshift-service-ca" pod "76b7c060-0a4a-4edb-a9c3-c89badeb1285" (UID: "76b7c060-0a4a-4edb-a9c3-c89badeb1285"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:02:03.004906 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:02:03.004877    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76b7c060-0a4a-4edb-a9c3-c89badeb1285-maas-api-tls" pod "76b7c060-0a4a-4edb-a9c3-c89badeb1285" (UID: "76b7c060-0a4a-4edb-a9c3-c89badeb1285"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:02:03.005237 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:02:03.005221    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76b7c060-0a4a-4edb-a9c3-c89badeb1285-kube-api-access-smxzq" pod "76b7c060-0a4a-4edb-a9c3-c89badeb1285" (UID: "76b7c060-0a4a-4edb-a9c3-c89badeb1285"). InnerVolumeSpecName "kube-api-access-smxzq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:02:03.005369 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:02:03.005348    5907 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76b7c060-0a4a-4edb-a9c3-c89badeb1285-metrics-certs" pod "76b7c060-0a4a-4edb-a9c3-c89badeb1285" (UID: "76b7c060-0a4a-4edb-a9c3-c89badeb1285"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:02:03.104161 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:02:03.104071    5907 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76b7c060-0a4a-4edb-a9c3-c89badeb1285-maas-api-tls\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 20:02:03.104161 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:02:03.104107    5907 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smxzq\" (UniqueName: \"kubernetes.io/projected/76b7c060-0a4a-4edb-a9c3-c89badeb1285-kube-api-access-smxzq\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 20:02:03.104161 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:02:03.104117    5907 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/76b7c060-0a4a-4edb-a9c3-c89badeb1285-metrics-certs\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 20:02:03.104161 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:02:03.104126    5907 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76b7c060-0a4a-4edb-a9c3-c89badeb1285-openshift-service-ca\") on node \"ip-10-0-132-152.ec2.internal\" DevicePath \"\""
Oct 30 20:02:03.619146 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:02:03.619106    5907 generic.go:356] "Generic (PLEG): container finished" podID="76b7c060-0a4a-4edb-a9c3-c89badeb1285" containerID="bac1b08951dc397623ac34e9b2c8cd7dcc799df3b6b82bf22076e811e538e54b" exitCode=0
Oct 30 20:02:03.619834 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:02:03.619743    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5f5c657c85-jtpzc" event={"ID":"76b7c060-0a4a-4edb-a9c3-c89badeb1285","Type":"ContainerDied","Data":"bac1b08951dc397623ac34e9b2c8cd7dcc799df3b6b82bf22076e811e538e54b"}
Oct 30 20:02:03.620010 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:02:03.619848    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5f5c657c85-jtpzc" event={"ID":"76b7c060-0a4a-4edb-a9c3-c89badeb1285","Type":"ContainerDied","Data":"ee736df6d8005115da49890bf1ba400a2fae9dac25c67ccfb6c9a4f8df65c5d0"}
Oct 30 20:02:03.620010 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:02:03.619908    5907 scope.go:122] "RemoveContainer" containerID="bac1b08951dc397623ac34e9b2c8cd7dcc799df3b6b82bf22076e811e538e54b"
Oct 30 20:02:03.620154 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:02:03.620089    5907 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5f5c657c85-jtpzc"
Oct 30 20:02:03.634621 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:02:03.634598    5907 scope.go:122] "RemoveContainer" containerID="bac1b08951dc397623ac34e9b2c8cd7dcc799df3b6b82bf22076e811e538e54b"
Oct 30 20:02:03.634951 ip-10-0-132-152 kubenswrapper[5907]: E1030 20:02:03.634931    5907 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bac1b08951dc397623ac34e9b2c8cd7dcc799df3b6b82bf22076e811e538e54b\": container with ID starting with bac1b08951dc397623ac34e9b2c8cd7dcc799df3b6b82bf22076e811e538e54b not found: ID does not exist" containerID="bac1b08951dc397623ac34e9b2c8cd7dcc799df3b6b82bf22076e811e538e54b"
Oct 30 20:02:03.635020 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:02:03.634960    5907 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bac1b08951dc397623ac34e9b2c8cd7dcc799df3b6b82bf22076e811e538e54b"} err="failed to get container status \"bac1b08951dc397623ac34e9b2c8cd7dcc799df3b6b82bf22076e811e538e54b\": rpc error: code = NotFound desc = could not find container \"bac1b08951dc397623ac34e9b2c8cd7dcc799df3b6b82bf22076e811e538e54b\": container with ID starting with bac1b08951dc397623ac34e9b2c8cd7dcc799df3b6b82bf22076e811e538e54b not found: ID does not exist"
Oct 30 20:02:03.676874 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:02:03.676831    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5f5c657c85-jtpzc"]
Oct 30 20:02:03.690172 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:02:03.690136    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5f5c657c85-jtpzc"]
Oct 30 20:02:04.286476 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:02:04.286437    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="76b7c060-0a4a-4edb-a9c3-c89badeb1285" path="/var/lib/kubelet/pods/76b7c060-0a4a-4edb-a9c3-c89badeb1285/volumes"
Oct 30 20:02:05.528445 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:02:05.528410    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 20:02:17.501036 ip-10-0-132-152 kubenswrapper[5907]: E1030 20:02:17.500948    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 20:02:32.398668 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:02:32.398601    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 20:02:35.282914 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:02:35.282883    5907 scope.go:122] "RemoveContainer" containerID="57204452be7cd38d9f03c8dea2e9facdf562a2e4140e44043c685c5ff4df67b6"
Oct 30 20:02:35.291840 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:02:35.291820    5907 scope.go:122] "RemoveContainer" containerID="19b92d6b42c4989eedb36ae6911e462bae1ff5f83de90e2500e91f783fae0c52"
Oct 30 20:02:45.868391 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:02:45.868348    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 20:02:47.500031 ip-10-0-132-152 kubenswrapper[5907]: E1030 20:02:47.499996    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 20:03:17.499384 ip-10-0-132-152 kubenswrapper[5907]: E1030 20:03:17.499341    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 20:03:22.028980 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:03:22.028947    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-ldvpq"]
Oct 30 20:03:22.042057 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:03:22.042029    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-ldvpq"]
Oct 30 20:03:22.286961 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:03:22.286883    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b40bbeeb-a0e5-4909-8816-2b9e6054b8f7" path="/var/lib/kubelet/pods/b40bbeeb-a0e5-4909-8816-2b9e6054b8f7/volumes"
Oct 30 20:03:35.320002 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:03:35.319970    5907 scope.go:122] "RemoveContainer" containerID="256207638a28f7f9cb4e524d3e065efd11e7e40bc2d4d8e904cd100a30d30f2c"
Oct 30 20:03:44.035282 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:03:44.035172    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-6qj88"]
Oct 30 20:03:44.048381 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:03:44.048352    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-6qj88"]
Oct 30 20:03:44.299054 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:03:44.298951    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2e11d300-dd1f-418b-bad6-bec630bfd7fc" path="/var/lib/kubelet/pods/2e11d300-dd1f-418b-bad6-bec630bfd7fc/volumes"
Oct 30 20:03:47.501042 ip-10-0-132-152 kubenswrapper[5907]: E1030 20:03:47.501007    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 20:04:17.500306 ip-10-0-132-152 kubenswrapper[5907]: E1030 20:04:17.500263    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 20:04:35.344469 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:04:35.344437    5907 scope.go:122] "RemoveContainer" containerID="4e641e38b9c03309d1de809168a710ad24f2f96e588987735a6d212ed916ad7e"
Oct 30 20:04:35.707156 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:04:35.707114    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 20:04:42.676428 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:04:42.676385    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 20:04:47.047693 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:04:47.047650    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f62d239e-rf2dm"]
Oct 30 20:04:47.060632 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:04:47.060604    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f62d239e-rf2dm"]
Oct 30 20:04:47.499066 ip-10-0-132-152 kubenswrapper[5907]: E1030 20:04:47.499024    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 20:04:48.286759 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:04:48.286728    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="00286d05-f903-48ca-ad1a-1415e4629850" path="/var/lib/kubelet/pods/00286d05-f903-48ca-ad1a-1415e4629850/volumes"
Oct 30 20:04:50.677654 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:04:50.677618    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 20:05:08.693995 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:05:08.693904    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 20:05:17.499966 ip-10-0-132-152 kubenswrapper[5907]: E1030 20:05:17.499913    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 20:05:35.365801 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:05:35.365764    5907 scope.go:122] "RemoveContainer" containerID="f5b5b9f5756f42ca5d0e58c4ae53fdd9aa236baeecdf4269cd062764c1daac4b"
Oct 30 20:05:36.989051 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:05:36.989017    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-28zhj"]
Oct 30 20:05:47.499615 ip-10-0-132-152 kubenswrapper[5907]: E1030 20:05:47.499567    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 20:05:48.468903 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:05:48.468854    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6b9b54ffb7-2k2b5_e732b2b4-b2be-4435-9d3e-5c3e744ee661/manager/0.log"
Oct 30 20:05:49.107855 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:05:49.107818    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-9wds9_4547301a-e9a4-4ecf-8946-55b8ea7c54a9/proxy/0.log"
Oct 30 20:05:49.844614 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:05:49.844581    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-9dcd87b65-p9k2r_416b5aec-c0a5-48ea-9716-7e23bd98192c/maas-api/0.log"
Oct 30 20:05:50.485459 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:05:50.485429    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-3e31a506-2brf5_761a9956-10ee-4642-948d-3c34f238daa0/revoke-keys/0.log"
Oct 30 20:05:50.690370 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:05:50.690338    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-452b6d7b-sdj44_ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb/revoke-keys/0.log"
Oct 30 20:05:51.075415 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:05:51.075384    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-d2067b2c-9jjqm_744e211a-0530-4633-9629-d0d740e4684d/revoke-keys/0.log"
Oct 30 20:05:51.476018 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:05:51.475988    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-wjf8m_5c355944-4ac3-4538-9430-72cd55cb9d1e/postgres/0.log"
Oct 30 20:05:54.183233 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:05:54.183201    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-28zhj_3e6bddcf-4a27-42f8-bc2a-f68e038ab415/limitador/0.log"
Oct 30 20:05:55.364278 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:05:55.364250    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-zdlcg_b8b6d708-4cea-4eec-b78d-030047017ecc/discovery/0.log"
Oct 30 20:05:55.549411 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:05:55.549384    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58487bf9b4-h7x8l_872f33d5-d35d-4bbf-b063-748f8dee26af/kube-auth-proxy/0.log"
Oct 30 20:05:56.029192 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:05:56.029160    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-sdj44"]
Oct 30 20:05:56.043499 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:05:56.043463    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-sdj44"]
Oct 30 20:05:56.286250 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:05:56.286169    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb" path="/var/lib/kubelet/pods/ab7ceb8c-80b8-4791-b02b-1a0930a7dbfb/volumes"
Oct 30 20:05:56.367151 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:05:56.367119    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-9tdql_8db50859-98fc-49aa-806b-4fbed56b1d00/istio-proxy/0.log"
Oct 30 20:05:56.563945 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:05:56.563847    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-c2sv6_ffa8893a-530b-4232-8e6e-dbe34b30ccda/istio-proxy/0.log"
Oct 30 20:05:57.173690 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:05:57.173661    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-pdg78_bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3/istio-proxy/0.log"
Oct 30 20:05:57.552118 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:05:57.552033    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-s826m_86efaf93-7f28-4e6a-9add-c8bdd87c5261/istio-proxy/0.log"
Oct 30 20:05:58.304760 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:05:58.304730    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-d59pz_91ee2b9c-4412-415e-817d-d92df100c950/payload-processing/0.log"
Oct 30 20:05:59.496194 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:05:59.496162    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp_817de1af-900b-4fbe-b792-24ac2de8b171/main/0.log"
Oct 30 20:05:59.887924 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:05:59.887889    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-wd6m6_0b52bf40-f75b-465b-8947-10ea73874f85/main/0.log"
Oct 30 20:06:12.081031 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:06:12.080995    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6b9b54ffb7-2k2b5_e732b2b4-b2be-4435-9d3e-5c3e744ee661/manager/0.log"
Oct 30 20:06:12.724607 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:06:12.724576    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-9wds9_4547301a-e9a4-4ecf-8946-55b8ea7c54a9/proxy/0.log"
Oct 30 20:06:13.275388 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:06:13.275359    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-9dcd87b65-p9k2r_416b5aec-c0a5-48ea-9716-7e23bd98192c/maas-api/0.log"
Oct 30 20:06:13.897839 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:06:13.897806    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-3e31a506-2brf5_761a9956-10ee-4642-948d-3c34f238daa0/revoke-keys/0.log"
Oct 30 20:06:14.278378 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:06:14.278294    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-d2067b2c-9jjqm_744e211a-0530-4633-9629-d0d740e4684d/revoke-keys/0.log"
Oct 30 20:06:14.652375 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:06:14.652347    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-wjf8m_5c355944-4ac3-4538-9430-72cd55cb9d1e/postgres/0.log"
Oct 30 20:06:15.027598 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:06:15.027511    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2brf5"]
Oct 30 20:06:15.040698 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:06:15.040667    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2brf5"]
Oct 30 20:06:16.286574 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:06:16.286536    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="761a9956-10ee-4642-948d-3c34f238daa0" path="/var/lib/kubelet/pods/761a9956-10ee-4642-948d-3c34f238daa0/volumes"
Oct 30 20:06:16.796797 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:06:16.796766    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-28zhj_3e6bddcf-4a27-42f8-bc2a-f68e038ab415/limitador/0.log"
Oct 30 20:06:17.499777 ip-10-0-132-152 kubenswrapper[5907]: E1030 20:06:17.499735    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 20:06:17.788408 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:06:17.788326    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-zdlcg_b8b6d708-4cea-4eec-b78d-030047017ecc/discovery/0.log"
Oct 30 20:06:17.994767 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:06:17.994739    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58487bf9b4-h7x8l_872f33d5-d35d-4bbf-b063-748f8dee26af/kube-auth-proxy/0.log"
Oct 30 20:06:18.789953 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:06:18.789919    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-9tdql_8db50859-98fc-49aa-806b-4fbed56b1d00/istio-proxy/0.log"
Oct 30 20:06:18.985208 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:06:18.985125    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-c2sv6_ffa8893a-530b-4232-8e6e-dbe34b30ccda/istio-proxy/0.log"
Oct 30 20:06:19.603277 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:06:19.603250    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-pdg78_bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3/istio-proxy/0.log"
Oct 30 20:06:20.002726 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:06:20.002696    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-s826m_86efaf93-7f28-4e6a-9add-c8bdd87c5261/istio-proxy/0.log"
Oct 30 20:06:20.754637 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:06:20.754608    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-d59pz_91ee2b9c-4412-415e-817d-d92df100c950/payload-processing/0.log"
Oct 30 20:06:21.497622 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:06:21.497594    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-kfgsp_817de1af-900b-4fbe-b792-24ac2de8b171/main/0.log"
Oct 30 20:06:21.914367 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:06:21.914339    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-wd6m6_0b52bf40-f75b-465b-8947-10ea73874f85/main/0.log"
Oct 30 20:06:35.386360 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:06:35.386323    5907 scope.go:122] "RemoveContainer" containerID="ae6c0644544bcb0275f8a7444678eb8e5315b3103e34aa68db7a3ad108fb274d"
Oct 30 20:06:35.395042 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:06:35.395017    5907 scope.go:122] "RemoveContainer" containerID="c5389877a0b19e6b08dbec0206f4923eec46c7b4d4671ac83f411a661b1d29fd"
Oct 30 20:06:38.046775 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:06:38.046736    5907 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2067b2c-9jjqm"]
Oct 30 20:06:38.058821 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:06:38.058795    5907 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2067b2c-9jjqm"]
Oct 30 20:06:38.287065 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:06:38.286985    5907 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="744e211a-0530-4633-9629-d0d740e4684d" path="/var/lib/kubelet/pods/744e211a-0530-4633-9629-d0d740e4684d/volumes"
Oct 30 20:06:47.500478 ip-10-0-132-152 kubenswrapper[5907]: E1030 20:06:47.500423    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 20:06:56.963100 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:06:56.963066    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6b9b54ffb7-2k2b5_e732b2b4-b2be-4435-9d3e-5c3e744ee661/manager/0.log"
Oct 30 20:06:57.255891 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:06:57.255799    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6b9b54ffb7-2k2b5_e732b2b4-b2be-4435-9d3e-5c3e744ee661/manager/0.log"
Oct 30 20:06:58.000563 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:06:58.000527    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-d59pz_91ee2b9c-4412-415e-817d-d92df100c950/payload-processing/0.log"
Oct 30 20:06:58.191982 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:06:58.191953    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-d59pz_91ee2b9c-4412-415e-817d-d92df100c950/payload-processing/0.log"
Oct 30 20:07:17.500015 ip-10-0-132-152 kubenswrapper[5907]: E1030 20:07:17.499973    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 20:07:35.422533 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:35.422498    5907 scope.go:122] "RemoveContainer" containerID="93e6cd29d573b4921d7276d615b97d92d3d9f1edfae8123382d5067042ba170e"
Oct 30 20:07:35.431245 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:35.431223    5907 scope.go:122] "RemoveContainer" containerID="72a8cc1a33bac52f4d546e0e5c768e71b9e2e21fb36512d5401b84d000338a70"
Oct 30 20:07:35.439127 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:35.439109    5907 scope.go:122] "RemoveContainer" containerID="fe42d33becb85d12799a9b8906b07de001c61ebdfd0d90e325d1b7b2e8d77ae3"
Oct 30 20:07:41.506522 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:41.506491    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6b9b54ffb7-2k2b5_e732b2b4-b2be-4435-9d3e-5c3e744ee661/manager/0.log"
Oct 30 20:07:41.794189 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:41.794109    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6b9b54ffb7-2k2b5_e732b2b4-b2be-4435-9d3e-5c3e744ee661/manager/0.log"
Oct 30 20:07:42.545797 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:42.545770    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-d59pz_91ee2b9c-4412-415e-817d-d92df100c950/payload-processing/0.log"
Oct 30 20:07:42.742359 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:42.742328    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-d59pz_91ee2b9c-4412-415e-817d-d92df100c950/payload-processing/0.log"
Oct 30 20:07:47.501784 ip-10-0-132-152 kubenswrapper[5907]: E1030 20:07:47.501743    5907 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p8nq5" containerName="extractor"
Oct 30 20:07:47.865639 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:47.865609    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-z7mhx_725d2cce-2a75-426f-8522-d1ddfbabe1b6/global-pull-secret-syncer/0.log"
Oct 30 20:07:48.531781 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:48.531755    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-nw4jx_2c840c6b-6992-4f29-9023-a0ad99fdc543/konnectivity-agent/0.log"
Oct 30 20:07:48.753069 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:48.753042    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-152.ec2.internal_e58ff4b1131d98fcba47fc4e4f2a3553/haproxy/0.log"
Oct 30 20:07:55.526269 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:55.526233    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-28zhj_3e6bddcf-4a27-42f8-bc2a-f68e038ab415/limitador/0.log"
Oct 30 20:07:57.202912 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:57.202884    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_70c6b30c-9245-4fd7-908b-9fc3eef24d91/alertmanager/0.log"
Oct 30 20:07:57.296327 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:57.296293    5907 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9zq2q/perf-node-gather-daemonset-cwwtb"]
Oct 30 20:07:57.296778 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:57.296764    5907 cpu_manager.go:414] "containerMap: removing container" podUID="76b7c060-0a4a-4edb-a9c3-c89badeb1285" containerName="maas-api"
Oct 30 20:07:57.296820 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:57.296782    5907 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="76b7c060-0a4a-4edb-a9c3-c89badeb1285" containerName="maas-api"
Oct 30 20:07:57.296900 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:57.296890    5907 memory_manager.go:365] "RemoveStaleState removing state" podUID="76b7c060-0a4a-4edb-a9c3-c89badeb1285" containerName="maas-api"
Oct 30 20:07:57.300153 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:57.300133    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9zq2q/perf-node-gather-daemonset-cwwtb"
Oct 30 20:07:57.322045 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:57.319991    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_70c6b30c-9245-4fd7-908b-9fc3eef24d91/config-reloader/0.log"
Oct 30 20:07:57.335592 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:57.335564    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9zq2q\"/\"kube-root-ca.crt\""
Oct 30 20:07:57.335592 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:57.335564    5907 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9zq2q\"/\"openshift-service-ca.crt\""
Oct 30 20:07:57.335814 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:57.335576    5907 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9zq2q\"/\"default-dockercfg-rgcbq\""
Oct 30 20:07:57.337787 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:57.337762    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9zq2q/perf-node-gather-daemonset-cwwtb"]
Oct 30 20:07:57.415938 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:57.415904    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f041c66e-c0db-45e5-9473-8df725432a13-podres\") pod \"perf-node-gather-daemonset-cwwtb\" (UID: \"f041c66e-c0db-45e5-9473-8df725432a13\") " pod="openshift-must-gather-9zq2q/perf-node-gather-daemonset-cwwtb"
Oct 30 20:07:57.416104 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:57.415954    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qzdk\" (UniqueName: \"kubernetes.io/projected/f041c66e-c0db-45e5-9473-8df725432a13-kube-api-access-6qzdk\") pod \"perf-node-gather-daemonset-cwwtb\" (UID: \"f041c66e-c0db-45e5-9473-8df725432a13\") " pod="openshift-must-gather-9zq2q/perf-node-gather-daemonset-cwwtb"
Oct 30 20:07:57.416104 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:57.416034    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f041c66e-c0db-45e5-9473-8df725432a13-sys\") pod \"perf-node-gather-daemonset-cwwtb\" (UID: \"f041c66e-c0db-45e5-9473-8df725432a13\") " pod="openshift-must-gather-9zq2q/perf-node-gather-daemonset-cwwtb"
Oct 30 20:07:57.416104 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:57.416088    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f041c66e-c0db-45e5-9473-8df725432a13-lib-modules\") pod \"perf-node-gather-daemonset-cwwtb\" (UID: \"f041c66e-c0db-45e5-9473-8df725432a13\") " pod="openshift-must-gather-9zq2q/perf-node-gather-daemonset-cwwtb"
Oct 30 20:07:57.416222 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:57.416129    5907 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f041c66e-c0db-45e5-9473-8df725432a13-proc\") pod \"perf-node-gather-daemonset-cwwtb\" (UID: \"f041c66e-c0db-45e5-9473-8df725432a13\") " pod="openshift-must-gather-9zq2q/perf-node-gather-daemonset-cwwtb"
Oct 30 20:07:57.430851 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:57.430824    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_70c6b30c-9245-4fd7-908b-9fc3eef24d91/kube-rbac-proxy-web/0.log"
Oct 30 20:07:57.517842 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:57.517763    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f041c66e-c0db-45e5-9473-8df725432a13-podres\") pod \"perf-node-gather-daemonset-cwwtb\" (UID: \"f041c66e-c0db-45e5-9473-8df725432a13\") " pod="openshift-must-gather-9zq2q/perf-node-gather-daemonset-cwwtb"
Oct 30 20:07:57.517842 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:57.517816    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qzdk\" (UniqueName: \"kubernetes.io/projected/f041c66e-c0db-45e5-9473-8df725432a13-kube-api-access-6qzdk\") pod \"perf-node-gather-daemonset-cwwtb\" (UID: \"f041c66e-c0db-45e5-9473-8df725432a13\") " pod="openshift-must-gather-9zq2q/perf-node-gather-daemonset-cwwtb"
Oct 30 20:07:57.518098 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:57.517843    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f041c66e-c0db-45e5-9473-8df725432a13-sys\") pod \"perf-node-gather-daemonset-cwwtb\" (UID: \"f041c66e-c0db-45e5-9473-8df725432a13\") " pod="openshift-must-gather-9zq2q/perf-node-gather-daemonset-cwwtb"
Oct 30 20:07:57.518098 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:57.517893    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f041c66e-c0db-45e5-9473-8df725432a13-lib-modules\") pod \"perf-node-gather-daemonset-cwwtb\" (UID: \"f041c66e-c0db-45e5-9473-8df725432a13\") " pod="openshift-must-gather-9zq2q/perf-node-gather-daemonset-cwwtb"
Oct 30 20:07:57.518098 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:57.517901    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f041c66e-c0db-45e5-9473-8df725432a13-podres\") pod \"perf-node-gather-daemonset-cwwtb\" (UID: \"f041c66e-c0db-45e5-9473-8df725432a13\") " pod="openshift-must-gather-9zq2q/perf-node-gather-daemonset-cwwtb"
Oct 30 20:07:57.518098 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:57.517922    5907 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f041c66e-c0db-45e5-9473-8df725432a13-proc\") pod \"perf-node-gather-daemonset-cwwtb\" (UID: \"f041c66e-c0db-45e5-9473-8df725432a13\") " pod="openshift-must-gather-9zq2q/perf-node-gather-daemonset-cwwtb"
Oct 30 20:07:57.518098 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:57.517959    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f041c66e-c0db-45e5-9473-8df725432a13-proc\") pod \"perf-node-gather-daemonset-cwwtb\" (UID: \"f041c66e-c0db-45e5-9473-8df725432a13\") " pod="openshift-must-gather-9zq2q/perf-node-gather-daemonset-cwwtb"
Oct 30 20:07:57.518098 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:57.517967    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f041c66e-c0db-45e5-9473-8df725432a13-sys\") pod \"perf-node-gather-daemonset-cwwtb\" (UID: \"f041c66e-c0db-45e5-9473-8df725432a13\") " pod="openshift-must-gather-9zq2q/perf-node-gather-daemonset-cwwtb"
Oct 30 20:07:57.518098 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:57.518017    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f041c66e-c0db-45e5-9473-8df725432a13-lib-modules\") pod \"perf-node-gather-daemonset-cwwtb\" (UID: \"f041c66e-c0db-45e5-9473-8df725432a13\") " pod="openshift-must-gather-9zq2q/perf-node-gather-daemonset-cwwtb"
Oct 30 20:07:57.534740 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:57.534716    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_70c6b30c-9245-4fd7-908b-9fc3eef24d91/kube-rbac-proxy/0.log"
Oct 30 20:07:57.538700 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:57.538672    5907 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qzdk\" (UniqueName: \"kubernetes.io/projected/f041c66e-c0db-45e5-9473-8df725432a13-kube-api-access-6qzdk\") pod \"perf-node-gather-daemonset-cwwtb\" (UID: \"f041c66e-c0db-45e5-9473-8df725432a13\") " pod="openshift-must-gather-9zq2q/perf-node-gather-daemonset-cwwtb"
Oct 30 20:07:57.612885 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:57.612830    5907 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9zq2q/perf-node-gather-daemonset-cwwtb"
Oct 30 20:07:57.646107 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:57.646078    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_70c6b30c-9245-4fd7-908b-9fc3eef24d91/kube-rbac-proxy-metric/0.log"
Oct 30 20:07:57.747611 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:57.747584    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_70c6b30c-9245-4fd7-908b-9fc3eef24d91/prom-label-proxy/0.log"
Oct 30 20:07:57.849372 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:57.849292    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_70c6b30c-9245-4fd7-908b-9fc3eef24d91/init-config-reloader/0.log"
Oct 30 20:07:57.915586 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:57.915555    5907 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9zq2q/perf-node-gather-daemonset-cwwtb"]
Oct 30 20:07:57.918481 ip-10-0-132-152 kubenswrapper[5907]: W1030 20:07:57.918454    5907 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf041c66e_c0db_45e5_9473_8df725432a13.slice/crio-aacd5f925272e609f13eaec39d46e9d277ab49cb271e9231134ad124ec8939b9 WatchSource:0}: Error finding container aacd5f925272e609f13eaec39d46e9d277ab49cb271e9231134ad124ec8939b9: Status 404 returned error can't find the container with id aacd5f925272e609f13eaec39d46e9d277ab49cb271e9231134ad124ec8939b9
Oct 30 20:07:58.083945 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:58.083912    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-mlpqh_1ffad3bc-f14e-4132-b5ee-3ec84d009252/kube-state-metrics/0.log"
Oct 30 20:07:58.181699 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:58.181672    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-mlpqh_1ffad3bc-f14e-4132-b5ee-3ec84d009252/kube-rbac-proxy-main/0.log"
Oct 30 20:07:58.281885 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:58.281836    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-mlpqh_1ffad3bc-f14e-4132-b5ee-3ec84d009252/kube-rbac-proxy-self/0.log"
Oct 30 20:07:58.388159 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:58.388132    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6b87cd958c-prt2j_2ca12196-1929-4ece-a076-d36527b95ff1/metrics-server/0.log"
Oct 30 20:07:58.490099 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:58.490012    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-mffps_aa367dd7-da29-424e-9acc-7706f4377494/monitoring-plugin/0.log"
Oct 30 20:07:58.601343 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:58.601308    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9zq2q/perf-node-gather-daemonset-cwwtb" event={"ID":"f041c66e-c0db-45e5-9473-8df725432a13","Type":"ContainerStarted","Data":"ddd55b08f09ea0195ed407f7aca936e97a9780bb58eaf2144c123ca9490b33d4"}
Oct 30 20:07:58.601566 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:58.601360    5907 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9zq2q/perf-node-gather-daemonset-cwwtb" event={"ID":"f041c66e-c0db-45e5-9473-8df725432a13","Type":"ContainerStarted","Data":"aacd5f925272e609f13eaec39d46e9d277ab49cb271e9231134ad124ec8939b9"}
Oct 30 20:07:58.601566 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:58.601456    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9zq2q/perf-node-gather-daemonset-cwwtb"
Oct 30 20:07:58.646638 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:58.646581    5907 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-9zq2q/perf-node-gather-daemonset-cwwtb" podStartSLOduration=1.6465658090000002 podStartE2EDuration="1.646565809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:07:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:07:58.644433703 +0000 UTC m=+2790.487365643" watchObservedRunningTime="2026-10-30 20:07:58.646565809 +0000 UTC m=+2790.489497747"
Oct 30 20:07:59.295268 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:59.295234    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jqs6k_f1622f2e-319b-4259-b314-a4178b7875f0/node-exporter/0.log"
Oct 30 20:07:59.394507 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:59.394478    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jqs6k_f1622f2e-319b-4259-b314-a4178b7875f0/kube-rbac-proxy/0.log"
Oct 30 20:07:59.495146 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:59.495120    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jqs6k_f1622f2e-319b-4259-b314-a4178b7875f0/init-textfile/0.log"
Oct 30 20:07:59.605268 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:59.605185    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-8k92x_a4ed12fa-0026-4bf8-a99c-83b04caac774/kube-rbac-proxy-main/0.log"
Oct 30 20:07:59.706765 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:59.706738    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-8k92x_a4ed12fa-0026-4bf8-a99c-83b04caac774/kube-rbac-proxy-self/0.log"
Oct 30 20:07:59.806429 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:59.806400    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-8k92x_a4ed12fa-0026-4bf8-a99c-83b04caac774/openshift-state-metrics/0.log"
Oct 30 20:07:59.922873 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:07:59.922828    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_aa815a19-9652-4492-9d04-5aa25725131c/prometheus/0.log"
Oct 30 20:08:00.022218 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:00.022188    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_aa815a19-9652-4492-9d04-5aa25725131c/config-reloader/0.log"
Oct 30 20:08:00.123206 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:00.123177    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_aa815a19-9652-4492-9d04-5aa25725131c/thanos-sidecar/0.log"
Oct 30 20:08:00.225190 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:00.225115    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_aa815a19-9652-4492-9d04-5aa25725131c/kube-rbac-proxy-web/0.log"
Oct 30 20:08:00.325891 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:00.325849    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_aa815a19-9652-4492-9d04-5aa25725131c/kube-rbac-proxy/0.log"
Oct 30 20:08:00.429105 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:00.429073    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_aa815a19-9652-4492-9d04-5aa25725131c/kube-rbac-proxy-thanos/0.log"
Oct 30 20:08:00.531499 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:00.531429    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_aa815a19-9652-4492-9d04-5aa25725131c/init-config-reloader/0.log"
Oct 30 20:08:00.991225 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:00.991199    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-dbd6b58fb-r6mpp_7c27d265-67e1-424e-85ff-38a8c30bd547/telemeter-client/0.log"
Oct 30 20:08:01.096295 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:01.096253    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-dbd6b58fb-r6mpp_7c27d265-67e1-424e-85ff-38a8c30bd547/reload/0.log"
Oct 30 20:08:01.195721 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:01.195690    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-dbd6b58fb-r6mpp_7c27d265-67e1-424e-85ff-38a8c30bd547/kube-rbac-proxy/0.log"
Oct 30 20:08:01.304070 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:01.303991    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66d96d76c5-nmsds_3b05813e-ce3d-4e71-a64e-7417997c2988/thanos-query/0.log"
Oct 30 20:08:01.403662 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:01.403630    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66d96d76c5-nmsds_3b05813e-ce3d-4e71-a64e-7417997c2988/kube-rbac-proxy-web/0.log"
Oct 30 20:08:01.510333 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:01.510303    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66d96d76c5-nmsds_3b05813e-ce3d-4e71-a64e-7417997c2988/kube-rbac-proxy/0.log"
Oct 30 20:08:01.615093 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:01.615068    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66d96d76c5-nmsds_3b05813e-ce3d-4e71-a64e-7417997c2988/prom-label-proxy/0.log"
Oct 30 20:08:01.726896 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:01.726848    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66d96d76c5-nmsds_3b05813e-ce3d-4e71-a64e-7417997c2988/kube-rbac-proxy-rules/0.log"
Oct 30 20:08:01.834218 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:01.834192    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66d96d76c5-nmsds_3b05813e-ce3d-4e71-a64e-7417997c2988/kube-rbac-proxy-metrics/0.log"
Oct 30 20:08:04.617314 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:04.617288    5907 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9zq2q/perf-node-gather-daemonset-cwwtb"
Oct 30 20:08:05.040304 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:05.040223    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6c4d7556c9-lgj6h_adf839a8-8656-4196-a59f-292012c006f6/console/0.log"
Oct 30 20:08:08.932020 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:08.931937    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vn5fp_4b70ef66-ed5b-4917-8bca-36757fee37e3/dns/0.log"
Oct 30 20:08:09.031448 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:09.031421    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vn5fp_4b70ef66-ed5b-4917-8bca-36757fee37e3/kube-rbac-proxy/0.log"
Oct 30 20:08:09.239596 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:09.239523    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-f7tbh_b41606bf-e8ca-4fc6-8805-b7461187d65e/dns-node-resolver/0.log"
Oct 30 20:08:09.973635 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:09.973601    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-zdlcg_b8b6d708-4cea-4eec-b78d-030047017ecc/discovery/0.log"
Oct 30 20:08:10.005364 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:10.005339    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58487bf9b4-h7x8l_872f33d5-d35d-4bbf-b063-748f8dee26af/kube-auth-proxy/0.log"
Oct 30 20:08:10.262059 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:10.261969    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-9tdql_8db50859-98fc-49aa-806b-4fbed56b1d00/istio-proxy/0.log"
Oct 30 20:08:10.319642 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:10.319617    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-c2sv6_ffa8893a-530b-4232-8e6e-dbe34b30ccda/istio-proxy/0.log"
Oct 30 20:08:10.484478 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:10.484450    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-pdg78_bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3/istio-proxy/0.log"
Oct 30 20:08:10.608904 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:10.608815    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-s826m_86efaf93-7f28-4e6a-9add-c8bdd87c5261/istio-proxy/0.log"
Oct 30 20:08:10.713989 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:10.713961    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tgt5g_ba4ee546-c9c1-4573-b11f-6ba0621972d1/node-ca/0.log"
Oct 30 20:08:10.874795 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:10.874767    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-d59pz_91ee2b9c-4412-415e-817d-d92df100c950/payload-processing/0.log"
Oct 30 20:08:12.815144 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:12.815107    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-zdlcg_b8b6d708-4cea-4eec-b78d-030047017ecc/discovery/0.log"
Oct 30 20:08:13.465741 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:13.465706    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58487bf9b4-h7x8l_872f33d5-d35d-4bbf-b063-748f8dee26af/kube-auth-proxy/0.log"
Oct 30 20:08:13.922331 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:13.922299    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-9tdql_8db50859-98fc-49aa-806b-4fbed56b1d00/istio-proxy/0.log"
Oct 30 20:08:14.068692 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:14.068660    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-zdlcg_b8b6d708-4cea-4eec-b78d-030047017ecc/discovery/0.log"
Oct 30 20:08:14.103823 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:14.103793    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-c2sv6_ffa8893a-530b-4232-8e6e-dbe34b30ccda/istio-proxy/0.log"
Oct 30 20:08:14.146327 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:14.146298    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58487bf9b4-h7x8l_872f33d5-d35d-4bbf-b063-748f8dee26af/kube-auth-proxy/0.log"
Oct 30 20:08:14.403295 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:14.403266    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-9tdql_8db50859-98fc-49aa-806b-4fbed56b1d00/istio-proxy/0.log"
Oct 30 20:08:14.430232 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:14.430210    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-pdg78_bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3/istio-proxy/0.log"
Oct 30 20:08:14.468131 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:14.468107    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-c2sv6_ffa8893a-530b-4232-8e6e-dbe34b30ccda/istio-proxy/0.log"
Oct 30 20:08:14.638568 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:14.638541    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-pdg78_bf9df589-b7ee-40ae-acdd-c0c1fc5eccf3/istio-proxy/0.log"
Oct 30 20:08:14.655321 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:14.655252    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-s826m_86efaf93-7f28-4e6a-9add-c8bdd87c5261/istio-proxy/0.log"
Oct 30 20:08:14.768834 ip-10-0-132-152 kubenswrapper[5907]: I1030 20:08:14.768799    5907 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-s826m_86efaf93-7f28-4e6a-9add-c8bdd87c5261/istio-proxy/0.log"